Inference Cost Is an Architecture Problem Most AI engineers I know have never seriously thought about inference cost until it destroyed their unit economics in production. I’ve watched it happen more times than I’d like to admit. The pattern is always the same: weeks of benchmarking, heated Slack debates about GPT-4o versus Claude versus Gemini,…
The Real Reason Your AI System Is Broken (It’s Not the Model) Every week I watch engineers spend hours debating which model to use. GPT-4o versus Claude 3.5 versus Gemini 1.5 Pro. Benchmark comparisons, pricing calculators, latency tests. It’s not that those things don’t matter. They do, at the margins. But in my experience shipping…
The Failure Mode Nobody Is Logging Most production AI systems don’t fail loudly. They don’t throw exceptions, trigger alerts, or produce obviously wrong outputs. They drift. Quietly, confidently, step by step, until you’re ten downstream tasks deep and the original intent is completely unrecognizable. I’ve been running multi-agent pipelines for over a year, and this…
Three Hours vs. Two Minutes I’ve been building ML systems long enough to have opinions about most things. Distributed training, gradient accumulation bugs, flaky data pipelines. I’ve seen enough failure modes that I stopped being surprised by them. Then I spent three hours staring at a distributed training failure last month and came up completely…
Most AI agents are blind. They see screenshots. They guess at selectors. They retry five times when a button moves two pixels. That is not a browser. That is a blindfolded person poking at a touchscreen. The dev-browser approach changes the mental model entirely. Instead of giving an AI agent a camera pointed at Chrome,…
The Supply Chain Attack Nobody Is Talking About Enough Everyone is watching Optimus videos this week. I get it. The robot stuff is genuinely impressive. But I keep coming back to something far more unsettling, something that directly affects every team shipping AI products right now. TeamPCP just ran a multi-stage supply chain attack across…
Speed Is Not a Strategy: Why Fast AI Development Is a Liability Dressed Up as a Virtue There is a pattern I keep watching repeat itself in AI engineering, and I am tired of pretending it is not a problem. The teams shipping fastest are often the teams with the least understanding of what they…
The Thread We Were Hanging By Every AI engineer reading this should stop what they are doing and pay attention. This week, Andrej Karpathy flagged one of the most alarming supply chain attacks I have seen target our specific corner of the software world. The litellm package on PyPI was poisoned. A single pip install…
The PyPI Dependency Trap Nobody Wants to Talk About Last week, ML engineers got a very clear look at how fragile the tooling ecosystem really is. A poisoned PyPI release of litellm, version 1.82.8, sat live on the registry for less than an hour. In that window, it was fully capable of exfiltrating SSH keys,…
The litellm supply chain attack this week should be a wake-up call for every ML engineer. One poisoned PyPI release. Less than an hour live. And it had the potential to exfiltrate SSH keys, AWS credentials, Kubernetes configs, API keys, crypto wallets, and shell history from every machine that ran pip install litellm or anything…
