Why I spend my time on local AI

Everyone's doing “AI transformation.” Startups, big corporates, doesn't matter the size. That part is loud and obvious. What nobody says out loud is that it doesn't land on every team the same way.

Tech teams are mostly fine. Almost everyone I talk to is already paying for Claude, Copilot, a few Codex seats (and honestly, I still meet people who have never actually tried building anything with AI, which surprises me every time). On my own team we jumped on Amazon's AI-DLC back in December 2025, and I've spent a stupid amount of time since then trying to make that workflow tighter.

The part that actually changed how I think is everywhere else in the building. Go talk to someone in finance, or HR, or ops. Their “AI” is a ChatGPT tab that's been open in the browser since Monday. And the second a company has any real data sensitivity, even that gets shut down, because obviously nobody is going to paste the company's numbers into a chat box and pray. So the work that happens a thousand times a week, the work sitting right on top of the most sensitive data, gets left out completely.

I saw this up close at one of the biggest FMCG companies in Turkey. Usage was basically zero. Then I started thinking about banks, brokerages, insurance, anyone who actually answers to a regulator, and it clicked. This is where local AI belongs. The problem was never access to the smartest model. It's that nobody has put something safe and usable in front of these teams.

Two things really bother me about the “just use the cloud” answer.

First is control. You're handing your most sensitive data and your actual decisions to someone else's servers, someone else's pricing, someone else's rules. (I wrote that sentence and immediately pictured saying it to a couple of boards I've sat in front of. Nicest way I can put it: it sounds like pulling the trigger on yourself.)

Second is dependency. If you build everything on top of something you can't swap out, you're one bad morning away from being stuck. They cut you off. They stop serving your country. They triple the price. They change the terms in a paragraph you didn't read. And if your whole operation runs on that, you're left with nothing. The tool that works perfectly today is the one you can't log into tomorrow. Local AI isn't a nice-to-have. It's insurance.

And here's the thing: for the boring, repetitive stuff, HR, finance, internal ops, you don't even have to take that risk. None of it needs a frontier model. A small setup, trained on your own data, running inside your own walls, is more than enough. The data never leaves. The cost doesn't move. You don't owe anybody anything. That, to me, is what local AI is really about. Taking control back.

And I'm not saying this from a chair. For a few weeks now I've been training and running my own models on a DGX Spark. Right now it's running Qwen3.6-35B-A3B-heretic-NVFP4-DFlash.

Here's what surprised me. When it's just you, or a small team, and you're not fighting a hundred requests at once, a model this size is genuinely enough. No cluster. No bill that grows every month. No data leaving the room. One box on a desk does most of what a company actually needs day to day. Fast enough, private, fixed cost, and mine.

None of this is new for me, by the way. Rhizio, the second brain I've been building, was already pointing this way. It lives in the cloud, but I built it to never care which provider you use: your own API key, whether that's OpenAI, Anthropic, Google, or a local Ollama, and your data staying with you as plain git and markdown. “Don't get locked in, own your own head” was the idea from the first commit. Local AI is just the next knot on the same rope.

What I actually believe is this. The advantage over the next few years won't go to whoever rents the biggest model. Anyone with a credit card gets the speed. It goes to whoever builds and runs the human side, the organization side, and their own infrastructure the best. Speed is for sale. Meaning and ownership aren't.

So I spend my evenings building a homelab. A Spark cluster, a couple of Mac minis, some switches, cables everywhere. Slowly putting together my own sovereign AI stack. The easy half of this whole thing, the tech team, is basically done. The hard half, the valuable one, getting a safe on-prem intelligence in front of the people who work with sensitive data every day, is barely starting. That's the half I'm betting on.