Handing the easy tickets to a free local model cost 1.65x more

The pitch is obvious enough that most people building with coding agents have had it. You've got a paid frontier model doing everything, and a backlog tail of small mechanical tickets that don't need it. So put a free local model on the tail. The expensive model writes a tight spec once, the local one grinds out the edit, deterministic gates catch the mess, you review a PR at the end.

I built that. It's called ticketbot: a Go CLI that takes a ticket, drives aider against a local Ollama model in a throwaway git worktree, runs a gate chain, and opens a branch for review. Then I benchmarked whether it saved anything, on 4 July 2026, across three real tickets from a live app's backlog.

The setup, so you can weigh the result against your own:

"Small local model" does a lot of work in this pitch, and it isn't what I ran. 36B total parameters across 256 experts, 8 of them active per token, so about 3B active - which is where the speed comes from, and why it fits on a 24 GB card at Q4_K_M at all. It does tools, vision and thinking. A capable model doing a mechanical job, not a toy punching above itself. That matters for how far the result generalises: if the harness loses with this model in the seat, a smaller one doesn't rescue it.

It cost 1.65x what having Claude do the same three tickets directly cost. All three of the local model's outputs were unmergeable. Not one ticket came out ahead.

Here's the measurement, why the premise doesn't hold, and the routing rule I got out of it - which is the only part I'd actually reuse.

What was being measured

Local compute is free, so the only question is Claude tokens. Three activities, each run as an isolated Sonnet subagent in its own git worktree, each measured with the harness's own subagent_tokens counter (exact, not a proxy, and it includes the agentic context that balloons across turns):

• A - Claude does the ticket directly. Explore, edit, build until green.
• B - Claude writes the taskspec for the local model. No implementation.
• C - Claude reviews the diff ticketbot produced, as a PR it's accountable for.

The harness wins only if B + C < A.

For closed tickets I branched the worktree off the fix's parent commit, so each agent saw a genuine pre-fix codebase rather than one where the answer was already sitting in the file it was about to open.

Three tickets off the same app - a website builder, ~30 open issues at the time:

• #34, easy. Sanitise error messages through a mapping layer. The ticket already named the file and the fix.
• #33, medium. An N+1 query in a listing page.
• #2, hard. Per-account site quota.

The numbers

116,040 tokens worse off across three tickets, for three diffs I couldn't merge.

And that's a floor, not the real cost. All three outputs came back FIX_NEEDED, so the real path adds a fix-or-redo pass on top of every row. Two of the three Claude-written specs also had authoring bugs that broke ticketbot before the local model ever ran (an invalid YAML escape, a reference to a test file that didn't exist, and a tests: field holding shell commands) - debugging those is unmeasured extra B.

Why it doesn't work

The thesis was that the expensive reasoning happens once, up front, and everything downstream runs on free compute. The "once" part is right. The "cheap" part isn't.

Writing a spec the local model can execute means finding the file, finding the API, reading the surrounding pattern and then spelling all of it out precisely. That is the same exploration as doing the fix, plus a writing step. B on its own ran 0.66x to 1.8x the cost of A. So the harness moves the typing to free compute, keeps the understanding on Claude, and then pays for that understanding twice - once to write the spec, once to review the output.

The tells were specific:

#34 - the ticket already named the file and the change, so implementing it directly (29k) was cheaper than writing a spec about it (53k). When a ticket is already a good brief, the spec step is pure overhead.

#33 - the spec author read the same library internals the direct agent read, to find the same batched API. Same understanding, done twice, billed twice.

#2 - both Claude agents independently found that the quota needed AssertLimit plus a registration call in mainTwadSetup.go. The local model produced a tidy single-file version that read the limit and enforced nothing. It compiled. It passed vet. It passed the tests. It would have merged as a security control that wasn't a control.

That last one is the whole reason review can't be cheap.

Green gates prove compile and structure, never semantics

The gate chain is deterministic and free, and it's genuinely good at what it does. In order, fail-fast:

1. scope - the model may only touch files the taskspec listed (aider with --yes-always will happily add any file the model names, so this is load-bearing)
2. gofmt on changed Go files
3. go build -o /dev/null ./...
4. go vet ./...
5. go test on the spec's packages, or the packages of the edited files

Five green gates and a plausible diff still told me nothing about whether the quota was enforced. Every gate I can automate is a structural gate, so the semantic check stays human (or stays Claude), which puts the expensive activity back in the loop for exactly the tickets where you most want it.

The fair objection, and why it doesn't rescue it

Claude files most of these tickets in the first place, so some of the spec exploration is already sunk at filing time. If B is free, the comparison becomes "Claude fulfils the ticket" against "Claude reviews the local model's attempt":

A 29% win, concentrated on the bigger tickets. Two corrections take it away.

Review isn't the cost - fix-to-mergeable is. I took #33, the single most favourable ticket, all the way to something I'd actually merge:

So the best case, on the best ticket, under the most generous assumption available, is 19%. Pay for the spec as well and the same ticket loses half again.

And "specs are free" is fictional exactly where it matters. #33 only wins if its spec is free, but its fix needed discovery the ticket didn't contain: which batched API, plus a dependency bump. The fix agent spent a good chunk of its 66k re-discovering the same dependency bump the direct agent had found. The local model wrote the easy loop rewrite and got the load-bearing part wrong.

The squeeze

There's a pincer here, and it's the part that generalises past my setup.

If a ticket already contains the fix, the spec is genuinely free - but the ticket is small, so review and fix overhead costs more than just doing it.

If a ticket needs discovery, that discovery lands on the expensive model regardless of where you put it - in the spec, in the review, or in the fix - and the local model only contributes the mechanical scaffold.

A win needs a ticket that's simultaneously big, fix-known and trivially verifiable. That's a codemod. And a codemod is better done by a script.

The routing rule

This is the reusable output, and it's the thing I'd hand to anyone about to build the same harness. Hand a ticket to a weak-model harness only if all five hold:

1. Mechanical, not semantic. Applying a known pattern, not reasoning about correctness, security, auth or data integrity.
2. Zero discovery. The exact change is in the ticket - which file, which API, which pattern. No "find the right function".
3. Self-contained. One or two files. No cross-module wiring, no registration step, no codegen.
4. Trivially verifiable, low blast radius. Correctness is obvious from the diff plus gates, and being wrong is cheap.
5. Execution-dominant. Enough repetitive typing that doing it by hand costs more than reviewing a machine's attempt.

Any one red flag and it stays with the expensive model: security or auth, data integrity, needs discovery, cross-module registration, UX judgement, or simply being small.

I ran the ~30-ticket backlog through that filter. Roughly 11 were security or correctness hardening, 3 data-integrity, 10 UX and editor work, 3 large features, 1 performance. Zero qualified. The only mechanical residue was thumbnail CSS and placeholder copy, both small enough that review overhead loses on its own.

Which is the uncomfortable bit. The filter is sound, and on a real backlog it fires on nothing.

What I'd have needed to change the answer

Tuning would have moved the wrong number. --auto-test, a CONVENTIONS.md, architect mode, a stronger local model - all of that improves mechanical execution quality, which was already the cheap part. None of it touches B, and B is the dominant killer.

Limits of what I measured, since they're real: single-shot per ticket, one local model, one Go codebase, three tickets, and the retry loop and finalisers came after the benchmark rather than before it. A retry loop would cut the FIX_NEEDED rate. It wouldn't cut the spec cost or the semantic review cost, which is why I didn't re-run it.

Two things worth keeping if you build this anyway

Ollama silently truncates, and there are three context numbers in play. The model advertises 262,144. Ollama's own default, when nothing pins it, is 2,048. And the tag I was actually serving carried PARAMETER num_ctx 4096 baked into its Modelfile, so the real ceiling was 4,096 - set by a line I'd written weeks earlier and forgotten, and 64x under what the model claims. Nothing warns you at any point in that chain. Edits just come back quietly wrong.

Check with ollama show <tag> --modelfile before you trust a context window, and pin it per request rather than per tag. ticketbot writes an aider model-settings file per run rather than trusting anything:

- name: ollama_chat/fastqwen:latest
  extra_params:
    num_ctx: 32768

Thinking models parrot the few-shot examples. Under aider's --edit-format whole, the local model kept emitting aider's own example edit blocks back at it and producing no actual edit. --edit-format diff fixed it. Model-specific, probably, but the failure looks like the harness being broken rather than the format being wrong, so it cost an evening.

The full invocation, for anyone reproducing this:

aider --model ollama_chat/<model> --model-settings-file <generated> \
      --edit-format diff --yes-always --no-auto-commit --no-check-update \
      --analytics-disable --no-stream --no-pretty --map-tokens 0 \
      --read <context files> --message <the spec> <editable files>

--map-tokens 0 because the file set is explicit - there's no point paying for a repo map when the spec already names every file the model may touch.

Where this leaves the idea

The general pattern is "cheap tier does the work, expensive tier supervises", and it only pays off if supervision is cheap. Supervision is cheap only when the output is trivially trustable, which needs either a trivially verifiable task or a model good enough that you'd have used it directly. I don't have a way out of that catch-22 and I don't think it's an implementation detail of mine.

Underneath it: the cost of a code change is understanding it, not typing it. Any scheme that offloads the typing and keeps the understanding on the expensive tier will pay for the understanding twice.

There are two versions of this that I didn't rule out, because the benchmark doesn't reach them. Take Claude out of B entirely - a human writes the spec - and the Claude cost is review only, which is where the 29% column above lives. But the human still has to understand the code to write a correct spec, so that buys machine time with the scarcer resource. It's a human-leverage tool at that point, not a token saver, and it should be pitched and measured as one. Or amortise one spec across many near-identical tickets, which is the only place the spec cost genuinely goes away. None of my sample qualified, and the shape of a backlog that would is a codemod's backlog.

ticketbot is shelved rather than deleted - M0 through M4 got built for completeness, and it's sitting there if a boilerplate-at-volume ticket class ever shows up. The M0/M1 build cost about a day and bought the answer before M2-M4 got written, which is the only bit of the process I'd repeat exactly.

If you've made the cheap-tier handoff pay on real tickets, I'd like to know what your ticket class looks like, because I couldn't find one.