It’s Not the Model, It’s the Harness: What Actually Makes Engineering Teams Faster With AI

Evgeny Nasonov
Evgeny Nasonov
It’s Not the Model, It’s the Harness: What Actually Makes Engineering Teams Faster With AI

Our CEO Vadim Peskov recently joined GoodFirms for an episode of their Conversations That Matter podcast. The question on the table is one we hear from almost every engineering leader we work with: the company bought AI coding tools for everyone, developers say they feel faster, so why isn't the team shipping faster?

Vadim's short answer: typing was never the bottleneck. Here is the longer version from the engineering side, with the numbers we see on real projects. You can watch the full episode on YouTube.

Typing was never the bottleneck

Code completion makes the part of the job that was already fast even faster. The pile-up just moves to the next step: review, integration, testing and, above all, the decision about what to build. Deciding what to build and how to build it is the hard part of software. Code is the last step.

That is why every Diffco project runs on a spec-driven methodology. Before an agent writes a line, our internal harness walks the team through the questions that matter for that client and that industry: who the user is, what done looks like, what the system already does, and what an auditor will ask.

AI is a wonderful yes man

Ask a model whether your idea is good and it will usually say yes. As Vadim put it on the show, "AI by default is a wonderful yes man." A good senior engineer does the opposite and pushes back on weak requirements before they turn into expensive code.

So we build our systems to say no by default. An idea has to survive questioning before it becomes a ticket.

On real systems, context beats model choice

Single-prompt generation works on a small feature in a clean codebase. Real products are different. They have acceptance criteria, a product direction, years of decisions nobody wrote down, and technical debt that someone took on for a reason.

On brownfield systems, results depend on how deliberately you manage context. We pair multi-agent frameworks with knowledge graphs, so agents can work across context measured in hundreds of millions of tokens and understand why the system looks the way it does.

Security by design is now the practical option

Security is the first concern people raise about autonomous agents, and it should be. Secure-by-design work used to double the price of a project, or more. Today we run a separate set of security-focused agents next to the build agents, with human review at the points that matter. Reading every line by hand is no longer the realistic control.

Building compliance in from day one is now fast enough that skipping it rarely makes sense. HIPAA and SOC 2 work follows established playbooks, while stricter regimes such as FedRAMP need a different approach.

What gains are realistic

These are the ranges Vadim shared on the podcast, from our own projects:

Project typeTypical gain
New feature builds20-30X
Complex systems moving in the right direction5-10X
Heavy legal, privacy or security requirements (for example FedRAMP)2-3X

Two examples from the episode. A Series B startup with about 80 developers and a four-year-old system is rebuilding its platform with four people, including the CTO and one product person, and plans to grow to 15-20 engineers. Its gain is closer to 30X.

An enterprise with 250 developers and a 15-20-year-old system was not looking to cut staff. It wanted acceleration, and it went from about 50% gains with AI tools to 7-10X once the process changed. Same team, different process.

The common thread: processes designed for humans hold AI back. The gains came from redesigning how work flows, not from a better subscription.

The senior engineer's new job

When agents do the heavy lifting, the valuable skill shifts. A senior engineer now needs to understand what matters to the client, and how LLMs, RAG and graphs actually behave. When something breaks, the first question is whether the ticket, the request or the partner input was wrong. The code is the last place to look.

An engineer who can't explain why they chose an approach is a risk. That is how teams end up with software nobody understands.

It's not the model, it's the harness

Vadim's one piece of advice for engineering managers: "It's not models, it's always harness." Models will keep changing every few months. The harness is what you control: specs, memory, security agents, and a weekly habit of testing and improving the pipeline. On the show, Vadim mentioned dropping a tool from our pipeline in April and retesting it the very next week. That cadence is the point.

If you want to see how this works in practice, here is how Diffco builds software with our AI SDLC methodology.

Three questions for your own team

  1. Who reviews the spec before anything gets built?
  2. How do your agents get context on your legacy systems?
  3. Can someone explain every architectural choice in plain words?

If any answer is "not sure", that is where your speed is hiding.

Want a second opinion on your setup? Book a free 30-minute architecture review with our team. We will look at your harness, not just your tools.

Keep reading

Explore our development services