Buy the Harness, Build the Verification

Bu yazının Türkçesi: Agent altyapısını hazır al, doğrulamayı kendin yaz

In August, DeepSeek open-sourced its agent harness under the MIT license. The model adapter, the tool registry, the session log and the agent loop itself are all plugins you can swap out. NVIDIA’s NeMo Agent Toolkit is Apache-licensed. Moonshot ships Kimi Work, a desktop agent that acts on your own machine. The part most engineers picture when they say “agent” is being given away by the companies that train the models.

So if you’re planning to build your own agent framework, my advice is: don’t. Take a harness off the shelf and spend the effort on the two things nobody will hand you: the tools that connect the agent to your domain, and the code that checks whether it did the right thing.

“Agent” is four layers, not one thing

The build-or-buy question has a different answer at each layer:

• Harness. The agent loop, tool dispatch, session log, sandbox, permissions, context compaction, model adapters, the UI. Buy it. Competition has driven its price to zero.
• Tools. MCP servers, adapters to your domain’s systems, the code that actually does things. Build these. This is the work.
• Orchestration. Retries, durable state, pausing for a human’s approval and resuming. Mostly buy, with a thin layer of your own.
• Product. The thing someone pays for. Build it.

The DeepSeek design is itself a tell. “No privileged core, everything is a plugin” is the shape infrastructure takes on its way to becoming a commodity.

Two tests a custom framework fails

I use a simple test to judge whether a piece of software will be absorbed by the big platforms. Four questions:

1. Is it generic, or does it need domain context to work?
2. Does it depend on data that’s hard to get, behind a physical or contractual door?
3. If it’s wrong, is anyone liable? Does it need certification?
4. Does it need physical or legacy integration: sensors, PLCs, on-site installs, old protocols?

Every answer of “generic, no data, no liability, pure software” counts against it. A custom agent framework fails all four. It works for everyone, it holds no data, nobody sues over a bad agent loop, and it’s pure software. Zero out of four: you’d be competing against free, from the frontier labs, indefinitely.

The second test is harsher. If the model were free, instant and perfect, would anyone still need your harness? No. Take the model away and a harness is a for-loop with error handling.

The instinct is right, the remedy is wrong

The usual reason for building your own is a good one: you don’t want to be locked into a language, a library or a vendor. But a framework only you maintain is the deepest lock-in there is.

The fix is to choose where the seam sits. The pattern I’ve settled on is portable core, disposable shell:

• Core libraries in plain Python or TypeScript, with zero agent-framework imports and a command-line entry point.
• Tools as thin MCP adapters over those libraries, around 20 lines each by my estimate.
• Skills and prompts as markdown files.
• Workflows that call the libraries directly, with no model in the deterministic steps.
• The harness as configuration, replaceable in an afternoon.

The one rule: the core library never knows an agent exists. Then the same code ships four ways: as an MCP tool, a CLI, a cron job, or a library inside someone else’s system. Switching harnesses costs you a config file, not a rewrite.

What’s left to build is verification

Here’s how I split a content pipeline between code and model. Code owns source ingestion, deduplication, novelty against what’s already been published, frontmatter, cross-posting and link checks. The model owns the angle, the draft and the rewrite for each platform.

Two items on the code side matter more than the rest: checking that every claim has a source, and checking that the draft still sounds like its author. The same pattern shows up in research, as citation integrity and as contradiction detection against the notes you already have. All of it is verification, and almost nobody builds it. “Every claim has a source, and this sounds like you” is the line between content and slop.

That’s why I think the transferable skill in this field is evals, not agent loops. Anyone can build a loop now. Almost nobody can tell you whether it was right.

The obvious first build is usually the wrong one

For a personal knowledge base, the obvious first build is an MCP server with tools like read_note and search_notes. It’s mostly wrong. Desktop agents already get read-write access to a folder, and cat plus ripgrep covers most of what those tools would do. Wrapping them is ceremony.

What survives is capability rather than access:

• Enforcing conventions on write: templates, frontmatter, folder placement, links, index rows. Other machinery depends on that output, so 95% right is a bug, and the failures are silent.
• Semantic search: ripgrep finds a word only if someone typed it.
• Graph queries: backlinks, orphans, stale items, contradictions between notes.

Even then, start with the cheap 80%: a rules file the agent reads before it writes. A prompt gets you most of the compliance. Write the code only when fixing the misses by hand becomes a chore.

Where this is wrong

There is a legitimate case for custom: air-gapped or regulated deployments, where no commercial harness can be audited or certified and you have to own the whole evidence chain. Look at what gets built there, though. Not a harness, but a narrow loop with three tools, no dynamic planning and a full audit log: a few hundred lines, not a framework. In industrial settings a fixed workflow with model calls at specific steps beats an open-ended agent loop almost every time, because it’s auditable, testable and bounded in cost.

Reading one small harness end to end is also worth doing, and so is writing a loop from scratch once, so it stops being magic. Just don’t ship it.

What I’m doing instead

My plan is to take one workflow, from research to a finished note, and build it as a plain library with a CLI and no framework. Then I’ll build its eval harness: fixtures, a scorer and a regression suite. The loop I’ll borrow. The part that tells me whether the loop was right is the part worth owning.