PyPI supply chain attack via litellm and the dependency risk problem in ML engineering
The litellm supply chain attack this week should be a wake-up call for every ML engineer.
One poisoned PyPI release. Less than an hour live. And it had the potential to exfiltrate SSH keys, AWS credentials, Kubernetes configs, API keys, crypto wallets, and shell history from every machine that ran pip install litellm or anything that pulled it in as a dependency.
97 million downloads a month. The blast radius on that is hard to overstate.
What stopped it? The attacker’s own bug crashed a developer’s machine by consuming all its RAM. A lucky accident, not a defense.
Karpathy made a point worth sitting with: classical software engineering treats dependencies as virtuous. We’ve been told small, composable packages are good. Reuse is good. Don’t reinvent wheels.
But that philosophy quietly transferred enormous risk into every project. You’re not just trusting the library you installed. You’re trusting its full dependency tree, every maintainer in that tree, and every account that has publish access to every one of those packages.
For ML projects specifically, this risk is worse than average. Our dependency graphs are massive. A typical ML service might pull in hundreds of transitive packages. Each one is a potential attack surface.
I’ve been thinking about what actually changes behavior here.
Private mirrors with pinned, audited packages help. Hash verification at install time helps. But neither of those is the default. Most teams ship without them.
Karpathy’s take is to use LLMs to write the functionality directly when it’s simple enough, avoiding the dependency entirely. That sounds extreme until you think about what you’re actually protecting against.
For a small utility function? Copy the logic. Vendor it. Own it.
For something complex like litellm itself? You probably need it. But you should pin the exact version, verify the hash, and treat any update as a deliberate decision, not a background process.
The real lesson is not “don’t use open source.” It’s that dependency hygiene has to become as standard in ML workflows as model versioning or data validation.
We’re careful about what goes into our models. We should be just as careful about what runs on our machines.
#MachineLearning #MLEngineering #CyberSecurity #Python #AIEngineering #SoftwareEngineering
