Remote vs local AI harness and model
I’m interested in moving more of my AI setup to local models and local agents. Part of my thinking about this is considering where data lives and where to consider keeping private data isolated from cloud providers.
Here’s a chart to help visualize where harnesses run, models infer, and data lives based on local vs remote:
The top left corner is the only place where everything is completely private and local.
Siri, for me, is the interesting one here, and it’s clear that Apple, ever the privacy-focused company, is thinking a lot about how to do inference locally and to keep your data private and on your devices (rather than in the cloud). Their Private Cloud Compute is designed to give people access to more computing power since compute/inference may be limited on your own devices, and PCC lets you take advantage of the beefier setup Apple can provide without losing privacy.
I mostly hang around the top right corner, where I spend my days chit-chatting with Claude Code. I’m still sending data about me each time I send a prompt to Claude. I’d much prefer not to, and hope to be able to stay in the top left corner or at least use PCC to do more of the work instead (c’mon, Apple!).
I’ve got friends who swear by the bottom right. And, hey, you get free compute from the big AI providers in their epic data centers with your plans, plus it’s a lot easier to not have to worry about keeping your laptop open to make sure your agents are still running.
I think we’re going to end up with a world where we have a mix of these. The trickier thing from a UX standpoint is how to help the user delineate when they’re sharing private data vs data that’s OK to send to a 3rd party provider. I think Apple’s already well suited for this, as they’ve been dealing with this when handling permissions for things like Contacts, Location, etc. The hard part may be in thinking how to protect the user in macOS more since agents on the Mac can pretty much read anything people give them access to.