The underrated engineering skill of knowing when to stop building and remove complexity instead
The Discipline Nobody Teaches
Every engineer I know is tracking the wrong thing. Lines of code shipped. Features deployed. Model latency shaved by another 12 milliseconds. These are easy to measure, which is probably why we measure them. But the skill that actually separates good AI engineers from great ones is almost impossible to put on a dashboard. It’s knowing when to stop.
I’ve watched teams spend three months fine-tuning a transformer for a classification task that a three-line regex would have solved. I’ve seen agent pipelines rebuilt from scratch because the team kept asking “how do we make this smarter” instead of “should this exist at all.” The compulsion to keep building is often the exact thing that kills a project.
Why AI Makes This Worse
The tools are fast. The demos are impressive. You spin up a prototype in an afternoon and it works well enough that momentum takes over. Before long you have an orchestration layer, a vector database, a fine-tuned retrieval model, and a custom evaluation harness, all in service of something that could have been a lookup table.
Andrej Karpathy said something on a recent No Priors podcast that stuck with me. He hasn’t typed a line of code since December. His default workflow is now expressing intent to persistent agents running for 16-plus hours a day. That’s genuinely interesting, but it also means the surface area for over-engineering has exploded. When building is that frictionless, the decision to not build requires active discipline.
The Complexity Debt Nobody Talks About
Technical debt from adding features is well understood. Complexity debt from failing to remove them is not. Every layer you add to a system is a layer that future-you has to debug, monitor, explain, and maintain. In ML systems this compounds fast. A pipeline that does too many things becomes a pipeline where nobody knows which thing is causing the regression.
I think about the 80/20 principle constantly in this context. Not as a cliche, but as a forcing function. If 80 percent of the value comes from 20 percent of the system, the honest engineering question is whether the other 80 percent of the system is worth its carrying cost. Usually it isn’t.
What “Stopping” Actually Looks Like
Stopping doesn’t mean giving up. It means asking a different set of questions before you write another line.
Is the complexity I’m about to add solving a real problem, or am I solving for the elegance of the solution? Is there a dumber version of this that ships in two days and gets 90 percent of the way there? If I deleted this component tomorrow, what would actually break?
The best technical decision I’ve made in the last year was deleting an entire agent subgraph that took a week to build. The simpler path without it was faster, cheaper, and easier to reason about. That week of work made the system worse. Admitting that and acting on it was harder than building the next feature would have been.
🔧
The Orchestration Trap
Karpathy’s framing from that same podcast is that the limits on AI systems right now aren’t model capability, they’re orchestration skill. The people operating at the top are the ones who know how to direct agents well. I agree with this. But orchestration skill without the judgment to prune is just elaborate scaffolding. You can be very good at running a complicated system that shouldn’t be complicated.
The engineers I respect most are the ones who treat deletion as a first-class engineering activity. They have a bias toward the boring solution. They default to less. They’re willing to look unproductive in the short term because they’re measuring for something a dashboard can’t capture.
That’s the skill. And almost nobody is teaching it.
The best time to ask whether you should keep building is before you start. The second best time is right now.
Sources & Further Reading
#AIEngineering #SoftwareEngineering #MachineLearning #TechLeadership #BuildingWithAI
