Writing a Coding Agent From Scratch

I wanted to know how a coding agent actually works, so I wrote one on Vertex AI. Not a wrapper over an existing agent SDK, but the loop, the tools, the permission model, the REPL, session persistence, slash commands, subagents, hooks, an MCP client, and context compaction.

Vertex hands you a model endpoint and authentication. Everything else is yours, which turned out to be the useful part. Every feature I had been treating as ambient platform magic had to be built, and building it is how you find out it was never magic.

The loop is the easy bit

Call the model, get back either text or tool calls, execute the tools, append the results, call again. That is an afternoon.

The rest of the year is in the questions the loop doesn't ask:

• What happens when a tool call is dangerous?
• What happens when the conversation exceeds the window mid-task?
• What happens when the user hits Ctrl-C halfway through a file write?
• What happens on the eightieth iteration, when the model is confidently looping?

Every one of those is a policy question wearing an engineering costume, and none of them has a right answer you can look up.

Permissions are a mode, not a prompt

The design I landed on has four permission modes rather than a per-call yes/no: a default that asks, one that auto-accepts edits, a plan mode that refuses to write anything at all, and a bypass for when you know what you're doing and have a clean git tree.

The insight was that "should I approve this?" is almost never a per-call decision. It is a decision about the session: what kind of work this is, how much you trust the current plan, whether you're watching. Asking per call trains the user to hit yes without reading, which is worse than not asking.

Configuration precedence, written down once

Settings resolve lowest to highest. Defaults, then user config, then project config, then project-local config, then environment, then CLI flags.

That chain is boring and it is load-bearing. Each layer exists because something has a different natural owner. The model belongs to the user, the permission rules belong to the repo, the region belongs to the machine, and a flag belongs to this one invocation. Get the order wrong and the fix is a special case, and special cases in a config resolver metastasise.

Compaction is the interesting one

The window fills. You have to summarise and continue, and where you cut determines whether the agent keeps working or quietly starts repeating itself.

What survives has to include the parts a summariser naturally drops: decisions that were already made and argued about, the exact error string, the file paths currently in play. A summary that reads well and drops the specifics feels correct right up until the agent re-proposes something you rejected twenty minutes ago.

I do not think I have solved this. I do think that having to implement it, rather than receiving it as a feature, is the single thing that most changed how I use these tools.