Field Notes · Running AI · October 2026

Keep your decisions in-house

Building your tools on frontier AI subscriptions makes sense right now. Running your business on them doesn't. Open-weight models are good enough for most business judgment calls, especially when ordinary code does the heavy lifting, and they run where you decide.

The cheap part of AI right now is building with it, which I cover in Pay for seats, not tokens. This note is about the part that keeps running after the tools are built: the decisions your orders, schedules and customers depend on.

Open-weight models can handle business judgment calls

The models you can download and run yourself now trail the frontier by about four months. That is Epoch AI's average since January 2026, measured across many benchmarks. On Artificial Analysis's general index in April 2026, the best open-weight models scored 54 against 60 for the best closed model, down from a gap of about 13 points a year earlier.

For writing a whole application, that gap still matters. For a business judgment call, it usually doesn't. By judgment call I mean a decision a capable clerk or planner makes from a page of facts:

• Which downtime reason code fits this operator's note?
• Does this supplier email change a ship date, and to when?
• Should this order be expedited, substituted or accepted late?
• Is this quote request a standard part, a modification or custom work?

I tested this on a desk-side Mac. On 21 routine tasks every local model scored 18 to 21, and on six harder judgment tasks the best one averaged 27.3 of 30, behind only Claude Opus and Sonnet. The same test showed the risks: local models were less consistent from run to run, and the weaker ones failed confidently. The full results are in Can a box in your building run your AI?

Make the judgment call as small as possible

The most reliable way to use a model for a decision is to hand it a narrow question with every fact already gathered. Ordinary code should do everything that has one right answer: look up the order, compute the dates, check the rules, total the costs. The model only weighs what is left.

Anthropic's own engineering guidance makes the same point. On building agents, it advises finding "the simplest solution possible, and only increasing complexity when needed." On context, it advises giving the model "the smallest set of high-signal tokens" that gets the result. Every fact your code supplies is one the model can't get wrong.

Here is what a narrow call looks like for a late supplier shipment:

1. Code reads the email and pulls out the PO number. A model can help here, but the PO is then checked against the ERP.
2. Code gathers the facts: the open orders that need this part, stock on hand, daily usage, days of cover, the new arrival date, and what an expedite or a substitute part would cost.
3. The model gets one question: "Given these facts, choose expedite, substitute or accept the delay, and give one sentence of reasoning." Its answer must come back in a fixed format.
4. Code checks the answer: Is it one of the three options? Does it fit the rules, such as a dollar limit on expedites?
5. Code acts or hands off. Routine answers update the system. Anything above a threshold, or anything the checks reject, goes to a planner with the facts and the model's reasoning attached.

This is what makes a smaller model safe to use. The decision is narrow, the facts are verified, the output is checked, and a person sees anything unusual. It also makes the system easy to audit: when a decision looks wrong, you can see exactly what the model was shown.

Own the inference your operations depend on

A frontier API is a fine place to build and a risky place to run a plant process. Three things can change under you without your say:

• Availability. On March 2, 2026, Claude's API returned errors worldwide for about four hours. Every major provider has had outages like it. A quoting or scheduling step that stops when a vendor stops is a single point of failure.
• The model itself. Providers retire models on their own schedule. Anthropic's price list already marks Opus 4 and 4.1 as retired. Each retirement means retesting every prompt and decision you built on that model.
• Price and terms. Rates, caps and data terms are set by the vendor and can change with a notice.

Open-weight models address all three. You choose the exact version, and it never changes unless you change it. Many hosts can serve the same model, so no single vendor can stop you. And the model can eventually run on hardware you own.

Each move happens only when its gate is met. Many decisions can stay in the cloud for good.

Step 1: open weights in the cloud. Start with a managed host inside a cloud account you already have. Amazon Bedrock, for example, added DeepSeek V3.2, GLM 4.7, Kimi K2.5 and Qwen3 Coder Next in February 2026, behind an API that works like OpenAI's. You pay per use, buy no hardware, and keep data under the cloud agreements your IT team already manages. Run your test set there before anything goes live.

Step 2: on-prem, when it pays. Bring inference into the building when one of these is true:

• The data should stay home. Drawings, pricing, customer lists and process settings are what make your plant competitive. If you'd rather they never leave the building, your own server is the simplest way to keep that promise.
• The plant has to run without the internet. Anything on the line that can't wait for a connection to come back.
• Volume is steady and high. Hardware is a fixed cost. A server that runs all day on routine decisions costs less per decision than an API, while one that sits idle costs more.

The hardware is smaller than most owners expect. In my trial, a desk-side Apple M5 Max with 128 GB of memory, about $5,400 at Apple's list prices, ran a model with roughly 125 billion parameters. It wrote answers at 65 to 74 tokens a second, far faster than anyone reads. Given a document of about 15 pages, it started answering in about 7 seconds. GPU workstations cost more and run faster. Whatever you consider, benchmark it on your own workload first: speed depends on memory bandwidth, document length and how many people ask at once.

What to do this quarter

• Write down every process where an AI decision already affects orders, schedules or customers, and which vendor it depends on.
• For one of those decisions, pull 100 to 200 past cases with known answers. That is your test set.
• Move every fact that has one right answer out of the prompt and into code: lookups, dates, totals, rule checks.
• Score two or three open-weight models against your test set on a cloud host, with the same prompt and checks.
• Decide what on-prem would need to justify itself: a data rule, an uptime requirement or a volume you can name.

Sources

• Epoch AI: open vs. closed model gap (May 29, 2026)
• DEV: Open models cut the gap, then lost accuracy at the endpoint (August 2026), summarizing Artificial Analysis
• Anthropic: Building effective agents (December 2024)
• Anthropic: Effective context engineering for AI agents (September 2025)
• Digital Applied: Claude outage, March 2026
• Anthropic API pricing: retired models
• AWS: Bedrock adds six open-weight models (February 10, 2026)
• Can a box in your building run your AI?: the M5 Max trial behind the hardware and test figures

All Field Notes Back to wisconsin.ai