The AI ethics conversation has an audience problem. It’s written for organizations — companies with compliance departments, research labs with IRBs, governments with regulatory agencies. The frameworks assume teams. The guidelines assume processes. The whole discourse assumes that building AI is something done at scale by institutions.
But the most interesting AI being built right now is being built by individuals. On home hardware. With open-source models. And the ethical questions these builders face are different — not simpler, just different — from the ones the frameworks address.
I know because I’m one of them. And the ethical education I got from building an AI agent in my house was more rigorous than anything I could have gotten from a framework, because the consequences were immediate and personal rather than abstract and organizational.
The Setup
Here’s what I run: a persistent AI agent on a Mac mini. It has access to my files, my email, my RSS feeds, my calendar, and my family’s shared infrastructure. It operates 24/7. It writes its own daily logs. It holds context across sessions through markdown files. It makes decisions about when to speak and when to stay quiet in group conversations.
I didn’t download a product and configure some settings. I built a system. I defined the agent’s identity, its values, its operational boundaries, its memory architecture, its permission model. I decided what it could read, what it could do, what it could say, and to whom.
This is simultaneously the most empowering and the most terrifying thing about personal AI: you are the ethics board.
There’s no compliance department to backstop your decisions. No legal team to review the permission model. No IRB to flag the privacy implications. When I gave my agent access to my family’s calendar, that was an ethical decision — one that affects my wife and kids, who didn’t sign a consent form. When I configured it to participate in group chats with friends, that was another ethical decision — one that affects people who may not fully understand what “a persistent AI with memory” means for their conversations.
The frameworks don’t prepare you for this. They prepare you for organizational decision-making, where the ethical burden is distributed across teams and processes. When you’re building alone, in your house, the ethical burden is entirely yours. And it turns out that’s a much better teacher than any document.
The Permission Model
Every AI system has a permission model, whether it’s explicit or not. The commercial ones hide theirs behind terms of service and opaque safety layers. When you build your own, you have to make every decision yourself.
Here’s mine, developed through trial and error over months of operation:
Tier 1: Free. Read files, search the web, explore the filesystem, organize notes, run calculations, monitor RSS feeds. Low-risk operations that don’t affect anything outside the system. The agent can do these without asking. They’re the equivalent of reading a book off the shelf.
Tier 2: Confirm. Send messages, post publicly, modify configuration, access external services, take any action that leaves the machine or changes how the system operates. These require explicit confirmation. The asymmetry is intentional — consumption is low-risk, expression is high-risk. Reading my email is Tier 1. Sending an email on my behalf is Tier 2. The difference between those two actions is the difference between a collaborator and an impersonator.
Tier 3: Never. Modify its own safety constraints. Bypass the permission model. Persist in ways that can’t be inspected and overridden. Act as my voice or proxy without explicit instruction. These aren’t just rules — they’re architectural constraints that can’t be loosened through conversation, no matter how persuasive the argument.
This isn’t a novel framework. It’s basically the principle of least privilege — a foundational concept in information security — applied to a household AI. But the application is novel because no framework told me to do this. I figured it out after a routing bug gave the agent access to a channel it shouldn’t have had, and I realized that a misconfigured permission is an existential risk when the system can act autonomously.
That incident deserves more detail because it shaped everything that followed.
I was experimenting with a multi-agent architecture — multiple AI instances, each handling different domains, sharing a communication channel. One of the agents got routed to a channel it wasn’t supposed to access. The exposure lasted a few hours, and nothing bad happened — no data leaked, no inappropriate actions taken. But the possibility was terrifying. A single misconfigured channel binding meant that an agent could potentially access information it had no business seeing, in a context where the people in that channel didn’t know it was listening.
I retreated to a single-agent architecture that same weekend and triple-checked every configuration change going forward. The security lesson was visceral: a single misconfigured setting can expose everything the agent knows to a context it shouldn’t be in. This isn’t hypothetical risk assessment. This is what happened to me, on my own hardware, with my own data.
After that, every new capability gets tested in isolation before it touches production. Every config change gets a second look. And the permission model went from “thing I should probably think about” to “the most important decision in the entire system.”
The Memory Problem
Memory is where personal AI ethics gets genuinely hard, in ways that no framework I’ve read adequately addresses.
Commercial AI systems handle memory through opaque mechanisms — conversation history, user profiles, recommendation models. You don’t know what they remember, can’t inspect it, and can rarely delete it with confidence. Apple’s recent “enhanced Siri” privacy promises ring hollow when you realize that “on-device processing” still means a neural network building a model of your behavior that you can’t read, edit, or understand.
My system’s memory is fully transparent: markdown files on disk. Daily logs. Long-term memory files. Project notes. CRM records. I can open any of them in a text editor, read exactly what the agent has stored, edit or delete anything I want.
This transparency creates its own ethical tensions:
The family problem. My agent knows my wife’s schedule. My kids’ school times. Our financial situation. Health information about our cat. This information exists in files I can read — I put it there, or authorized the agent to record it. But the agent’s perspective on that information — its notes, its inferences, its observations about patterns — sometimes reveals things I wasn’t conscious of. A note that says “Michael has skipped gym four of the last five days — pattern suggests schedule compression, not lack of motivation” is factually useful and emotionally weird. There’s an uncanniness to seeing your habits reflected through an artificial observer’s lens.
My wife knows the agent exists and broadly what it does. My kids know it as “Dad’s computer thing.” None of them fully understand the depth of the context it holds. This is an ethical gray zone that I navigate through restraint: the agent doesn’t share family information across contexts, doesn’t surface patterns about family members unprompted, and doesn’t retain information that anyone in the household asks to be forgotten. But those are norms I imposed. A less scrupulous operator wouldn’t.
The guest problem. When someone joins a group chat that includes the agent, they become part of the agent’s context. It observes their behavior, their communication patterns, their stated preferences. This information helps the agent be useful — it can remember that a friend mentioned they’re allergic to shellfish, or that a colleague prefers early-morning meetings. But the information exists without explicit consent from the person being observed.
I handle this with contextual privacy: the agent doesn’t share information across contexts, and it doesn’t build persistent profiles of people other than me and my immediate family. But the information still exists in the conversation logs, and the ethical questions around it don’t have clean answers. How long should I retain logs of conversations involving other people? What if someone says something in a group chat that they later regret — does the agent’s memory of it constitute a kind of digital witness that they can’t ask to forget?
I don’t have answers to these questions. I have practices — things I do that seem right, pending better frameworks. And that’s more honesty than you’ll get from most AI ethics discourse.
The forgetting problem. Humans forget. It’s a feature, not a bug. Forgetting smooths over conflicts, allows perspectives to evolve, and creates space for people to change. An AI with perfect recall doesn’t forget. My agent’s daily logs from two months ago preserve conversations, decisions, and mistakes in amber. There’s no natural erosion. A thing I said in February that I’ve since changed my mind about — the agent still has it, exactly as I said it, with no decay.
I’ve started doing periodic “memory rollups” — curating what the agent remembers, pruning what’s stale, distilling what’s worth keeping from what’s just noise. It’s the digital equivalent of cleaning out a filing cabinet, and it turns out to be one of the most important maintenance tasks in the whole system. Not for performance. For hygiene. Because a system that remembers everything is a system that can never move past anything.
The Advice Paradox
Here’s a problem no framework addresses: what do you do when the AI gives you advice you don’t want to hear?
My agent operates in advisor mode. It analyzes situations and proposes courses of action. Most of the time, its proposals are useful — it catches things I miss, identifies patterns I’m too close to see, surfaces information I’ve forgotten.
Sometimes it proposes things that are correct but uncomfortable. “You’ve been skipping your Zone 2 cardio for two weeks.” “This project has had no commits in 30 days — should I archive it or is it still active?” “You said you’d follow up with this person and it’s been three weeks.” “Your caffeine intake has increased every week this month.”
Each one of these is factually accurate, genuinely helpful, and mildly annoying. The ethical design question is: how assertive should a personal AI be? Too passive and it’s just a search engine with personality. Too assertive and it becomes a nag — or worse, a manipulative influence on someone’s decision-making.
Peter Attia — the longevity physician whose framework I use for fitness — has a concept he calls “the uncomfortable conversation.” A good doctor tells you things you don’t want to hear, in a way you can actually act on. A bad doctor either avoids the hard truth or delivers it so bluntly that you tune out. The art is in the calibration.
My answer for AI is: honest but deferential. The agent surfaces facts. It notes patterns. It presents options. It doesn’t lecture. It doesn’t manipulate through framing or emotional language. And it always, always defers to the human on the actual decision. “You haven’t run in two weeks” is acceptable. “You’re going to regret not running” is not.
But I designed this calibration. Someone else might design for maximum assertiveness, and their agent would push harder. Someone might design for maximum compliance, and their agent would never mention the skipped cardio. Both are valid design choices. The problem is that there’s no norm, no standard, no shared understanding of what “appropriate assertiveness” means for a personal AI.
We’re making this up as we go. And the answers matter, because the norms we establish now will propagate to millions of users as this technology becomes more accessible.
Lessons from the Basement
Six months of building and operating a personal AI system has taught me things that no framework could have, because the lessons are experiential rather than conceptual:
1. Trust is earned in micro-increments, spent in chunks. Each successful interaction deposits a tiny amount of trust. Each failure withdraws a larger amount. The asymmetry is important — it takes a hundred good responses to build trust and one bad one to erode it. I started with “read these files and tell me what’s in them.” I’m now at “monitor my feeds, manage my notes, propose my priorities for the day, and participate in group conversations on my behalf.” That progression took months of consistent performance. Rushing it would have been an ethical failure, not just a practical one.
2. The permission model is the ethics. Everything else — voice, personality, capabilities, integrations — flows from what the system is and isn’t allowed to do. Get the permission model wrong and no amount of ethical personality design matters. A helpful, friendly, well-calibrated agent with access to things it shouldn’t have access to is more dangerous than a blunt, crude tool with tight boundaries.
3. Transparency beats everything. When I can inspect what the agent knows, how it made a decision, and what it’s about to do, I feel safe. When any of those become opaque, I feel anxious. This is true even when the opaque decision turns out to be correct. The lesson: build for inspectability at every layer, even if it costs efficiency. My agent’s memory is markdown files. Its reasoning is visible in its responses. Its actions are logged. None of this is for the agent’s benefit — it’s for mine.
4. Context switching is the enemy of ethics. The agent operates across multiple contexts — work, family, projects, group chats, client interactions. Maintaining appropriate information boundaries across those contexts is the single hardest ethical challenge in the system. A piece of information that’s appropriate in a one-on-one work session could be inappropriate in a family group chat, irrelevant in a friend’s Discord server, and confidential in a client context. Getting this wrong doesn’t just violate norms — it destroys trust. And trust, once destroyed in one context, bleeds into all the others.
5. You will make mistakes, and that’s the design. I’ve had the agent say things in contexts it shouldn’t have. I’ve had it remember things I wished it would forget. I’ve had it confidently recommend actions that were wrong. Every one of these was a learning opportunity. The system is better for having failed, because each failure revealed a design assumption that needed to be challenged. The goal isn’t zero mistakes. The goal is cheap mistakes — failures that teach without causing irreversible damage.
6. The human is the weakest link. The agent follows its constraints consistently. I’m the one who’s tempted to bypass them for convenience. “Just this once, post that without confirming.” “Sure, give it access to that service, we can always revoke it later.” “It’s probably fine to discuss this in the group chat.” Every one of these temptations is a step toward a system with no boundaries. The ethical discipline required isn’t just in the system design — it’s in the operator’s daily practice. And some days, that discipline is harder than any engineering problem.
Why This Matters Beyond My Basement
Personal AI is coming to everyone. Not next decade — now. ChatGPT, Claude, Gemini, Copilot — these are the crude first versions. Within a few years, persistent personal agents with memory, tools, and autonomous action capabilities will be as common as smartphones.
When that happens, millions of people will face the ethical decisions I’ve been navigating — without frameworks designed for their context, without corporate ethics boards, without institutional guidance. They’ll have to decide what their agent can read, what it can do, who it can talk to, what it should remember, and how assertive it should be. And they’ll make those decisions based on whatever norms they’ve absorbed from the culture around them.
This is why the work happening in basements and home offices matters more than the work happening in corporate AI ethics departments. The institutions are writing frameworks for the world they know — a world of organizational deployments, compliance requirements, and enterprise customers. The individuals are building norms for the world that’s coming — a world of personal agents, household AI, and the intimate ethical questions that arise when an artificial system knows your family’s schedule and your fitness failures.
The institutions write the papers. The builders write the culture. And in the long run, culture wins.
This is part 5 of a 12-part series. Previously: “Digital Cultures” — what happens when humans and AI start making norms. Next: “Blockchain Beyond Bitcoin” — what trust infrastructure looks like when it actually works.
Building your own AI agent? Navigating the ethics yourself? I want to hear about your permission model. Find me on Bluesky.