Build your own Jarvis: a brain, a memory, a voice and a team
Reel 46 · step-by-step guide · October 31, 2026
Four parts, and you can stop after any of them. The brain and the memory alone already give you most of the value.
The guide · 6 parts
- 01
Meet it first
Tap the ring to hear it. This is the line it says when my morning brief is ready.
Good morning, sir. Your brief is ready.
The real voice of my agent. The ring is drawn for this page and moves with how loud it speaks.
- 02
The brain: Claude Code
Install Claude Code and sign in. Unlike a chat window, it works inside a folder on your computer: it opens files, runs code and checks its own work until the task is done.
claude · ~/jarvis>
A real run on the morning brief: it reads the file, writes a script, runs it, and answers. - 03
The memory: your notes
Keep your notes in an Obsidian vault (a normal folder of Markdown files) and start Claude Code inside that folder. It can now search and read everything you have written, so it never starts from zero.
Then add a file called CLAUDE.md at the top of the vault. Claude Code reads it at the start of every session. Mine is the most important file in the whole setup. Start small:
- Write down decisions with the reason. That is what makes the answers personal.
- When it gets something wrong, fix CLAUDE.md, not just the answer.
467 notes, 1,850 links
Drag to turn
My real vault, one dot per note. Tap a folder to light it up. CLAUDE.md# Who I am Name, what I do, what I'm working on this year. # Goals The one goal everything else is judged against. # How to work with me - Answer with the conclusion first. - Don't guess numbers. Say "unknown" instead. - Ask before sending, posting, deleting or spending money. # Where things live - Projects/ one note per project, current state only - Decisions/ one note per decision, with the reason
- 04
The voice: ElevenLabs Agents
This part is separate from Claude Code. In ElevenLabs, create an agent, pick a voice, pick the model it thinks with (Claude is one of the options), and write a short system prompt: who you are and how it should talk. You can talk to it in the browser right away.
For a phone number, connect a Twilio number to the agent with ElevenLabs' native Twilio integration. Then you can call it, and it can call you.
- Pick a voice from the library or design one. Don't clone a real person's voice without their permission.
- Anyone who can reach the agent can talk to it unless you turn on authentication (signed URLs) or an allowlist. Do this before you connect anything private.
- To give it facts (your calendar, your sales), add tools that call your own small read-only API. Keep secrets and customer data out.
The phone side is its own part. It talks with Claude, but it can't read my notes yet. - 05
The team: subagents
Claude Code can hand work to subagents: each one is a Markdown file with its own role, rules and tools. Put them in ~/.claude/agents/ and they work from any folder. Mine are an analyst, a marketer and an engineer. I say what I want; Claude Code picks who does it.
- One role per file. A narrow description is what makes Claude Code pick the right one.
- Give each role only the tools it needs. The engineer can edit code; the analyst can only read.
Each role is one file with its own rules and tools. Tap a request to see who gets it. ~/.claude/agents/analyst.md--- name: analyst description: Data analyst. Use for "what changed", "is X working" and any numbers question. tools: Read, Grep, Glob, Bash --- You answer questions about my business numbers. - Start from the data, not from a guess. Say where each number came from. - Split averages by period to find the step changes. - Read-only: never change production data.
- 06
Bonus: the 6:30 brief
Every morning at 6:30 a scheduled job on my Mac (launchd) runs a small script that pulls the numbers I care about and saves a one-page brief, so it's ready before I wake up. It only runs on days the Mac is awake. Ask Claude Code to write the script and the schedule for you.
06:30, every day
Morning brief, Oct 30
Market numbers, closing prices
- S&P 5007,720.39-0.30%
- Nasdaq Composite26,184.72-1.38%
- Nikkei 22563,954.19+1.08%
- Dollar-yen157.67-0.20%
- US 10-year yield4.99%
Good to know
- The phone agent and Claude Code are two separate parts. In my setup the phone agent can't read my notes directly yet.
- ElevenLabs and Twilio are paid: ElevenLabs bills by call minutes (LLM usage is billed separately), Twilio for the number each month and for call minutes. Check each pricing page.
- An overview by @copilot_shogo. Not affiliated with Anthropic, Obsidian or ElevenLabs.
Free for people building with Claude Code
Two of us run a six‑figure business on this Claude Code setup.
Most setups forget who you are every session. This one knows your goal, pushes back when you drift, and checks its own work. Take the starter version. It takes 15 minutes to set up.
✻
CLAUDE.md
- START-HERE.md
- CLAUDE.md
- prompts/interview.md
- agents/verifier.md
- diagnostic.md
Confirm your email and the download starts. One email a week after that, unsubscribe any time.
MORE LIKE THIS