DEV Community: ImportStatic

PostgreSQL's recovery_prefetch prefetches pages the WAL already holds

ImportStatic — Mon, 02 Nov 2026 01:57:22 +0000

Originally published on ImportStatic.

A post on ardentperf from October 20, which reached the Hacker News front page, describes a synchronous standby on a 500 IOPS cloud disk that fell behind after a write burst and stalled writes on the primary for over 30 minutes. With recovery_prefetch = off the long stall went away, and the author leans towards turning it off everywhere. We don't have a standby on metered storage, so we couldn't reproduce the outage. We did count what the prefetcher spends its reads on during replay on PostgreSQL 18.6, and the same counters are available on your own standby.

Some of those reads are for pages that replay is about to rebuild from a full-page image in the WAL. The prefetcher leaves a block alone when the record that touches it carries an image, but it doesn't keep track of that block afterwards. When the next record touches the same page, it asks the operating system to read the page from disk. One heap page and two updates are enough to reproduce it, and the cause is a dozen lines of xlogprefetcher.c.

What the prefetcher does

recovery_prefetch arrived in PostgreSQL 15. The docs describe it as trying "to prefetch blocks that are referenced in the WAL that are not yet in the buffer pool, during recovery". The default is try, which turns it on wherever the OS can take read-ahead advice. On this Mac the PostgreSQL 18.6 build accepted on, which a build without read-ahead support rejects.

The prefetcher decodes WAL ahead of replay. In the REL_18_STABLE source, it allows maintenance_io_concurrency reads in flight (16 by default) and looks up to four times that many block references ahead, within the wal_decode_buffer_size window (512 kB). It only considers the main fork. For each block reference, the checks run in this order, and each outcome increments a column of pg_stat_recovery_prefetch:

Check	Counter
the record carries a full-page image of the block	`skip_fpw`
the block will be zero-initialised	`skip_init`
the relation or block doesn't exist yet	`skip_new`
the block is one of the last four blocks it considered	`skip_rep`
the block is already in shared buffers	`hit`
otherwise: read-ahead advice for one 8 kB block	`prefetch`

The "last four" window is updated after the full-page-image check. A block skipped for its image returns early and never enters the window, so the window can't catch the next reference to it. Shared buffers won't help either, because when the prefetcher reaches that later record, replay often hasn't restored the image yet.

Reproducing it with one page

The test table has three 1 kB rows per 8 kB page (fillfactor = 50, no indexes), so every update is a HOT update on the same page. After a CHECKPOINT, a Go driver ran these statements, each as its own transaction, then stopped the server with pg_ctl -m immediate and started it again so crash recovery replayed them:

update t set v = v + 1 where ctid = '(0,1)';
update t set v = v + 1 where ctid = '(1,1)';   -- pages 1 .. N, one row each
update t set v = v + 1 where ctid = '(2,1)';
update t set v = v + 1 where ctid = '(0,2)';   -- page 0 again

pg_get_wal_block_info from pg_walinspect lists the block references for t in WAL order. With 20 pages between the two visits to page 0, it printed:

0+fpi 1+fpi 2+fpi 3+fpi 4+fpi 5+fpi 6+fpi 7+fpi 8+fpi 9+fpi 10+fpi 11+fpi 12+fpi 13+fpi 14+fpi 15+fpi 16+fpi 17+fpi 18+fpi 19+fpi 20+fpi 0

The first touch of each page after the checkpoint carries a full-page image, as full_page_writes promises. The second touch of page 0 doesn't. After the crash, the counters were:

 prefetch | hit | skip_init | skip_new | skip_fpw | skip_rep
----------+-----+-----------+----------+----------+----------
        1 |   0 |         0 |        0 |       22 |        0

The single prefetch was for page 0. Every other block was skipped for its image; the 22 are the 21 pages of t plus one pg_attribute page that a hint-bit write had imaged. Replay restores page 0 from the image in the first record, so the second record finds it in shared buffers and never uses the read the prefetcher asked for.

With other gaps, and with full_page_writes off:

Pages between the two visits	`full_page_writes`	prefetch	hit	skip_fpw	skip_rep
0	on	1	0	4	0
2	on	1	0	4	0
20	on	1	0	22	0
2	off	4	0	0	1
20	off	22	1	0	0

With images on, page 0 is prefetched a second time even when the two records are back to back. With images off, the four-entry window catches the repeat at a gap of 2 (skip_rep = 1) but not at 20. The short runs count a few more images than t has because replay also covered catalog hint-bit images written after the test; the prefetch column is the one to read.

Stephen Frost raised this case during review in 2021, estimating thousands of unnecessary prefetch calls in a bad case with the default 512 kB decode buffer. He judged that the benefit outweighed the cost and that the redundancy could be fixed later. As of October 22, 2026, the REL_18_STABLE code still does the image check before it updates the window.

pgbench runs

We ran the same crash-and-count loop on pgbench data (scale 50, pgbench_accounts at 91,513 pages, 128 MB of shared buffers). "Bulk" updates 5% of the accounts in physical order in one statement. "Random" is 160,000 standard pgbench transactions from 8 clients. Each line is one run:

Workload	`full_page_writes`	WAL	prefetch	hit	skip_fpw	skip_rep
bulk	on	643 MB	99,192	28,363	82,502	777,355
bulk	off	71 MB	126,604	24,845	0	836,061
random	on	684 MB	143,991	442,138	79,855	242,243
random	off	67 MB	145,457	458,868	0	290,126

In the random workload the prefetcher skipped 79,855 blocks for their images and still issued only 1,466 fewer prefetches than with images off. The random runs touched different rows, so the two lines aren't an exact pair, but they agree with the single-page test, where skipping a page's first reference mostly moves the prefetch to its next reference. In the bulk case images cut prefetches by about a fifth.

Crash recovery took between 0.35 and 1.66 seconds in every run, on a laptop SSD with the files still in the OS page cache. Those times say nothing about a standby on a 500 IOPS volume.

Checking your own standby

The counters are cumulative from the last reset, so reset them before a write burst and read them after:

select pg_stat_reset_shared('recovery_prefetch');
-- ... later
select prefetch, hit, skip_fpw, skip_rep, io_depth from pg_stat_recovery_prefetch;

Every prefetch is read-ahead advice for a single 8 kB block. On local NVMe that's cheap, and the default is fine. On a volume with an IOPS cap, a standby whose prefetch count runs close to skip_fpw while replay lag grows is the case to test. We'd turn it off there and watch replay lag through the next burst.

The setting has sighup context: ALTER SYSTEM SET recovery_prefetch = off and SELECT pg_reload_conf() apply it without a restart, which we confirmed in the server log. Changing wal_decode_buffer_size needs a restart; maintenance_io_concurrency doesn't, and 0 disables prefetching too.

If you're planning the move to PostgreSQL 19, check this along with the rest of the upgrade checks: we only read the REL_18_STABLE prefetcher for this article.

Originally published on ImportStatic. This article was drafted with AI assistance; every claim was checked against the sources linked in it and the code samples were compiled and run.

anthropic-sdk-go 1.81 traces your Claude calls and forwards your OTel baggage

ImportStatic — Tue, 27 Oct 2026 14:45:38 +0000

Originally published on ImportStatic.

If your Go service registers a global OpenTelemetry tracer provider and propagator, upgrading to anthropic-sdk-go v1.81.0 (released October 14) changes two things with no code change: every Claude call becomes a span, and every request to the API carries the headers your propagator writes. In most Go services those are traceparent and baggage, since the OpenTelemetry Go getting-started guide registers both. Whatever your services keep in baggage now goes to Anthropic.

To see what goes over the wire, we sent the same request from v1.80.0 and v1.81.0 to an httptest server that returns canned Messages API responses. The request ran inside a handler span whose context held one baggage member, user.email=jane@example.com:

v1.80.0:  traceparent=""  baggage=""
          spans: "POST /tickets/summary"

v1.81.0:  traceparent="00-9cdfdc78baf5c4413a4de1c242b1b145-f6538546ab8336d9-01"
          baggage="user.email=jane@example.com"
          spans: "anthropic.messages.create" (client), "POST /tickets/summary"

No API key was used and nothing reached Anthropic, so this shows what the SDK sends and nothing about how the real API handles it.

What v1.81.0 changes

According to the release note, if the app registers a global tracer provider, "the SDK will create spans with it after upgrading", and "requests will also get trace context headers from the global propagator". The Claude Platform release notes for the same day list this under a beta of OpenTelemetry export, which also lets the API send its own server-side spans to your collector once an admin adds a destination in the Console.

The SDK side needs no setup with Anthropic. It reads otel.GetTracerProvider() and otel.GetTextMapPropagator(), so any service that calls otel.SetTracerProvider at startup is opted in.

Services that pass a provider explicitly to their instrumentation and never set the global one are only partly opted out. In our run of that setup (handler span from a non-global provider, global propagator registered), the SDK recorded no span of its own, but the request still went out with traceparent, carrying the handler span's ID, and the baggage. A comment in StartAPICallSpan covers this case: a non-recording span is still used when its trace context "goes on" in the headers. A process sends nothing only if it has neither a span in the context nor a registered propagator.

The feature is in beta. The SDK tracing guide warns that span names, attributes, options and environment variables can change between versions and tells you to pin the SDK version, which matters if you build dashboards on these attribute names.

The baggage header

The SDK guide mentions it once: "If that propagator also writes baggage, the SDK sends your baggage too." OpenTelemetry's baggage page warns that "sensitive Baggage items can be shared with unintended resources, like third-party APIs", and api.anthropic.com is now one of them.

Before this release, a Claude request only carried that header if you'd wrapped the SDK's HTTP client in otelhttp or set it yourself. With v1.81.0, any service that registered the composite propagator from the getting-started guide sends it:

otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(
    propagation.TraceContext{}, propagation.Baggage{}))

To know what leaves, check baggage extracted from incoming requests as well as what your own code sets. Upstream services add members too, so grepping for baggage.NewMember only finds part of it.

We tested three ways to keep traceparent and drop baggage:

Change	`traceparent` sent	`baggage` sent	Side effects
Register only `propagation.TraceContext{}`	yes	no	Global: every other client in the process stops sending baggage too
`option.WithOpenTelemetryPropagation(false)`	no	no	API-side spans can't join your trace
SDK middleware that deletes the header	yes	no	Only this client is affected; no help if the client uses `otelhttp`

The SDK injects trace headers before it runs your middleware chain, so a middleware can remove them. For a service whose other outbound calls need baggage, this is the one we'd use:

client := anthropic.NewClient(option.WithMiddleware(
    func(req *http.Request, next option.MiddlewareNext) (*http.Response, error) {
        req.Header.Del("baggage")
        return next(req)
    }))

If nothing else in the process reads baggage, registering only TraceContext{} is simpler, and it's the setup the SDK guide recommends.

What the span records

In the default metadata_only mode, a non-streaming call produced one CLIENT span named anthropic.messages.create under the handler span. Its attributes, with server.port, url.path and url.scheme left out and the port removed from url.full:

anthropic.message.stop_reason=end_turn
anthropic.request.id=req_test
gen_ai.operation.name=chat
gen_ai.provider.name=anthropic
gen_ai.request.max_tokens=256
gen_ai.request.model=claude-sonnet-5-5
gen_ai.response.id=msg_01
gen_ai.response.model=claude-sonnet-5-5
gen_ai.usage.input_tokens=12
gen_ai.usage.output_tokens=3
http.request.method=POST
http.response.status_code=200
http.route=/v1/messages
server.address=127.0.0.1
url.full=http://127.0.0.1/v1/messages

No prompt text is recorded in this mode. Setting ANTHROPIC_OPEN_TELEMETRY_TRACES_CONTENT_MODE=content added two attributes with the prompt and the reply; our test prompt included the email address:

gen_ai.input.messages=[{"role":"user","parts":[{"type":"text","content":"Summarise ticket 44...
gen_ai.output.messages=[{"role":"assistant","parts":[{"type":"text","content":"Hi"}]}]

The v1.81.0 source (internal/tracing/tracing.go) reads that variable on every call rather than once when the client is built, and an option passed to anthropic.NewClient overrides it. If prompts must never reach your tracing backend, pass option.WithOpenTelemetryTracesContentMode(option.OpenTelemetryContentModeMetadataOnly) explicitly so an env var in a deployment manifest can't change it.

Retries, otelhttp and streams

Retries. With the stand-in answering the first request with a 529 overloaded_error, the SDK retried using the same traceparent for both attempts and recorded one span with http.request.resend_count=1 and status Ok. The span reference documents this: one span covers the whole call, retries included, so failed attempts show up in that attribute and not in the span count.

otelhttp. With the SDK's HTTP client wrapped in otelhttp.NewTransport, we got the anthropic.messages.create span with an HTTP POST span under it, no duplicate anthropic.* span, and a single traceparent value at the server, carrying the otelhttp span's ID. The SDK guide suggests WithOpenTelemetryPropagation(false) in this setup. In our Go run the headers were identical with and without it, because otelhttp writes them last.

For the same reason the baggage middleware above has no effect here: otelhttp adds the header again after it runs. This setup already sent baggage before v1.81.0, and registering only TraceContext{} is the way to stop it. The HTTP POST span repeats the request timing; drop otelhttp from this client if you don't want it.

Streams. Messages.NewStreaming returns a stream whose span ends only when you've read it to the end or closed it:

read all events:         span ended, status Ok, gen_ai.response.time_to_first_chunk=0.0023
read 1 event, Close():   span ended, status Unset, no output token count
read 1 event, no Close:  no span exported

If your loop can stop before the last event, defer stream.Close(), or that call's span is never exported.

Counting tokens from these spans

With an API-side destination configured as well, each request produces two spans named anthropic.messages.create, yours (CLIENT) and the API's (SERVER), and both carry gen_ai.usage.*. The OpenTelemetry overview says to sum only one source per request, preferring the API spans when both exist; summing both doubles your token counts. The API also honours the sampled flag in traceparent, so per the API guide a trace your sampler drops gets no API spans either.

Span delivery is best effort, and the guide points to the Usage API for spend tracking. Our article on LLM API spend limits covers what the vendors enforce.

Turning it off

Pass option.WithoutOpenTelemetry() to disable it for one client, or set ANTHROPIC_OPEN_TELEMETRY=false for the whole process. With either, the server got no trace headers and only the handler span was recorded. Either is reasonable if another library already traces your Claude calls, or until you've checked what's in your baggage.

If you're upgrading across several SDK versions at once, the Go type changes from earlier releases still apply; we covered the ones that bit Haiku 5.5 migrations in Content[0].Text coming back empty.

Originally published on ImportStatic. This article was drafted with AI assistance; every claim was checked against the sources linked in it and the code samples were compiled and run.

Every OpenAI model shutdown date until January 2027, and what replaces it

ImportStatic — Sat, 24 Oct 2026 11:35:59 +0000

Originally published on ImportStatic.

If your code calls gpt-3.5-turbo, gpt-4, gpt-4-turbo, gpt-4.1-nano, o1, o1-pro, o3-mini, o4-mini or gpt-image-1, it has until October 23, 2026, eight days from today. On October 14 OpenAI also scheduled the GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini snapshots for shutdown on January 14, 2027, with gpt-6-luna as the replacement for all of them.

Everything below comes from OpenAI's deprecations page as of October 15, 2026. At shutdown, it says, "the model or endpoint will no longer be accessible". The page doesn't name the error you'll get, so plan for requests failing outright rather than being routed to another model.

Shutdown dates and replacements

Chat, reasoning and image models, ordered by shutdown date:

Shuts down	Model IDs	Replacement
2026-10-23	`gpt-3.5-turbo`, `gpt-3.5-turbo-0125`	`gpt-5.6-terra`
2026-10-23	`gpt-4`, `gpt-4-0613`, `gpt-4-1106-preview`, `gpt-4-turbo`, `gpt-4-turbo-2024-04-09`	`gpt-5.6-sol`
2026-10-23	`gpt-4o-2024-05-13`	`gpt-5.6-sol`
2026-10-23	`o1`, `o1-2024-12-17`, `o3-mini`, `o3-mini-2025-01-31`	`gpt-5.6-sol`
2026-10-23	`o1-pro`, `o1-pro-2025-03-19`	`gpt-5.6-sol` with `reasoning.mode: pro`
2026-10-23	`o4-mini`, `o4-mini-2025-04-16`	`gpt-5.6-terra`
2026-10-23	`gpt-4.1-nano`, `gpt-4.1-nano-2025-04-14`	`gpt-5.6-luna`
2026-10-23	`gpt-image-1`	`gpt-image-2.5-sunburst` or `gpt-image-2.5-flare`
2026-12-01	`gpt-image-1-mini`, `gpt-image-1.5`, `chatgpt-image-latest`	`gpt-image-2.5-sunburst` or `gpt-image-2.5-flare`
2026-12-11	`gpt-5-2025-08-07`, `o3-2025-04-16`	`gpt-5.6-sol`
2026-12-11	`gpt-5-pro-2025-10-06`, `o3-pro-2025-06-10`	`gpt-5.6-sol` with `reasoning.mode: pro`
2026-12-11	`gpt-5-mini-2025-08-07`	`gpt-5.6-terra`
2026-12-11	`gpt-5-nano-2025-08-07`	`gpt-5.6-luna`
2027-01-14	`gpt-4o-2024-08-06`, `gpt-4o-2024-11-20`, `gpt-4o-mini-2024-07-18`, `gpt-4.1-2025-04-14`, `gpt-4.1-mini-2025-04-14`	`gpt-6-luna`

The October 23 entry also lists completions-endpoint variants (gpt-3.5-turbo-completions, gpt-4-completions and others) and the fine-tuned snapshots ft-gpt-3.5-turbo, ft-gpt-4, ft-gpt-4.1-nano-2025-04-14, ft-o4-mini-2025-04-16, ft-babbage-002 and ft-davinci-002. The audio and realtime models (gpt-realtime, gpt-audio, gpt-4o-realtime, gpt-4o-mini-audio and five more) follow on January 20, 2027, and tts-1 and tts-1-hd on January 6, 2027.

Aliases without a date

For October 23 the page lists both the alias and the snapshot (o4-mini and o4-mini-2025-04-16). The December 11 and January 14 entries list only dated snapshots, and nothing on the page says what gpt-4o-mini, gpt-4o, gpt-4.1 or gpt-5 will resolve to once their snapshots are gone. openai-go v3.78.0 has made its own call for December and marks gpt-5, gpt-5-mini, gpt-5-nano and o3 as deprecated with a December 11 shutdown. We'd treat every alias as going away with its snapshot and move off it before then.

Fine-tuned GPT-4o mini and GPT-4.1 models

The January entry says "Fine-tuned versions of these models follow the same shutdown schedule as their underlying base models", so an ft:gpt-4o-mini-2024-07-18:... model stops answering on January 14.

The gpt-6-luna model page lists fine-tuning as unsupported, and the fine-tuning guide says OpenAI "is winding down the fine-tuning platform". According to the deprecations page, existing customers can't create new fine-tuning jobs after January 6, 2027. The only base models the guide still lists for supervised fine-tuning are the three GPT-4.1 snapshots, all of which are in the table above. A product that depends on a fine-tune has three months to replace it with a prompt (with examples) on gpt-6-luna, checked against the fine-tune's outputs on its own evaluation set.

Finding the model IDs in your code

openai-go marks deprecated models with Go's standard // Deprecated: comment, which staticcheck's SA1019 check reports. Our test service names models in the ways real code does (an SDK constant, a string constant, a map of fallbacks, an environment default and a YAML config), and staticcheck 2026.2.2 reports one of them:

$ go run honnef.co/go/tools/cmd/staticcheck@latest ./app/
app/main.go:18:15: github.com/openai/openai-go/v3.ChatModelO4Mini is deprecated: Announced shutdown date: 2026-10-23. See https://developers.openai.com/api/docs/deprecations for details and recommended replacements.  (SA1019)

The rest go unreported for two reasons. ChatModel is declared as type ChatModel = string, so a model written as a string literal is just a string, with nothing for SA1019 to attach to. And openai-go v3.78.0 came out on October 13, a day before the January deprecation was announced, so ChatModelGPT4oMini and ChatModelGPT4_1Mini have no deprecation marker yet. A later release will probably add one, but that won't help with string literals.

To catch those, we wrote a scanner that walks a tree, matches whole model IDs in .go, .py, .ts, .js, .yaml, .json, .toml and .env files, and resolves openai.ChatModel... constants by parsing shared/shared.go from the SDK's module directory. Its core is the shutdown table and a pattern that won't match gpt-4 inside gpt-4o or gpt-4.1:

func init() {
    add("2026-10-23", "gpt-5.6-terra", "gpt-3.5-turbo", "gpt-3.5-turbo-0125", "o4-mini", "o4-mini-2025-04-16")
    add("2026-10-23", "gpt-5.6-sol", "gpt-4", "gpt-4-0613", "gpt-4-1106-preview", "gpt-4-turbo",
        "gpt-4-turbo-2024-04-09", "gpt-4o-2024-05-13", "o1", "o1-2024-12-17", "o3-mini", "o3-mini-2025-01-31")
    add("2026-10-23", "gpt-5.6-luna", "gpt-4.1-nano", "gpt-4.1-nano-2025-04-14")
    // ... the rest of the table above ...
    add("2027-01-14", "gpt-6-luna", "gpt-4o-2024-08-06", "gpt-4o-2024-11-20", "gpt-4o-mini-2024-07-18",
        "gpt-4.1-2025-04-14", "gpt-4.1-mini-2025-04-14")
    alias("2027-01-14", "gpt-6-luna", "gpt-4o", "gpt-4o-mini", "gpt-4.1", "gpt-4.1-mini")
}

// idPattern matches a whole model ID: the characters around it must not be
// part of an ID, so "gpt-4" doesn't match inside "gpt-4o" or "gpt-4.1".
func idPattern() *regexp.Regexp {
    ids := make([]string, 0, len(models))
    for id := range models {
        ids = append(ids, regexp.QuoteMeta(id))
    }
    slices.SortFunc(ids, func(a, b string) int { return len(b) - len(a) })
    return regexp.MustCompile(`(?:^|[^\w.-])(` + strings.Join(ids, "|") + `)(?:$|[^\w.-])`)
}

On the same service it finds nine references and exits with status 1, so it can gate CI:

$ go run ./scan -sdk $(go list -m -f '{{.Dir}}' github.com/openai/openai-go/v3) ./app
app/config.yaml:2: literal gpt-4o-mini shuts down 2027-01-14 -> gpt-6-luna (alias; the page lists the snapshot)
app/config.yaml:3: literal gpt-4.1-2025-04-14 shuts down 2027-01-14 -> gpt-6-luna
app/config.yaml:4: literal gpt-image-1 shuts down 2026-10-23 -> gpt-image-2.5-sunburst or -flare
app/main.go:13: literal gpt-4o-mini-2024-07-18 shuts down 2027-01-14 -> gpt-6-luna
app/main.go:16: literal gpt-4.1-nano shuts down 2026-10-23 -> gpt-5.6-luna
app/main.go:17: constant ChatModelGPT4_1Mini = gpt-4.1-mini shuts down 2027-01-14 -> gpt-6-luna (alias; the page lists the snapshot)
app/main.go:18: constant ChatModelO4Mini = o4-mini shuts down 2026-10-23 -> gpt-5.6-terra
app/main.go:25: literal gpt-4o shuts down 2027-01-14 -> gpt-6-luna (alias; the page lists the snapshot)
app/main.go:30: constant ChatModelGPT4oMini = gpt-4o-mini shuts down 2027-01-14 -> gpt-6-luna (alias; the page lists the snapshot)
9 references

text-embedding-3-small on the next line of the YAML isn't on any list and isn't reported. A bare o1 in an unrelated YAML key would be a false positive. The scanner also can't see a model name that only exists in a deployed environment variable or a database row, so those need checking separately.

Moving a tool-calling gpt-4o-mini call to gpt-6-luna

A Chat Completions call that uses tools needs more than a new model string. The latest-model guide says "GPT-6 Sol and GPT-6 Luna support function calling in Chat Completions only with reasoning_effort: "none"", and Luna's default effort is medium. The same guide says to remove temperature, top_p and top_logprobs (and logprobs on Chat Completions) whenever effort isn't none. It doesn't say whether they're accepted at none, so we dropped temperature as well.

We built the old and new requests with openai-go and sent them to a local httptest stub that records the body; nothing went to OpenAI, since there's no API key on this machine. The new params look like this:

openai.ChatCompletionNewParams{
    Model:    openai.ChatModelGPT6Luna,
    Messages: []openai.ChatCompletionMessageParamUnion{openai.UserMessage("Weather in Oslo?")},
    Tools:    []openai.ChatCompletionToolUnionParam{weatherTool},
    // Function calling on Chat Completions needs "none" for gpt-6-luna.
    ReasoningEffort:     shared.ReasoningEffortNone,
    MaxCompletionTokens: openai.Int(500),
}

The stub recorded these bodies:

before (messages and tools omitted):
{
  "max_tokens": 500,
  "model": "gpt-4o-mini",
  "temperature": 0.2
}
after (messages and tools omitted):
{
  "max_completion_tokens": 500,
  "model": "gpt-6-luna",
  "reasoning_effort": "none"
}

The SDK's own field comment calls max_tokens deprecated in favour of max_completion_tokens, which also counts reasoning tokens. That comment isn't a // Deprecated: marker, so staticcheck didn't flag MaxTokens either. To have Luna reason and call tools in the same request, the guide says to move the call to the Responses API.

Price and context window

Per million tokens, gpt-6-luna lists $0.10 input and $0.50 output, against $0.15 and $0.60 for GPT-4o mini, and its context window is 1,050,000 tokens against 128,000, so cost is no reason to delay the move. For more on Luna's typed-answer endpoint, see our piece on the OpenAI Decisions API from Go. If you also call Claude, our Claude Sonnet 4.5 deprecation piece runs the same staticcheck test on anthropic-sdk-go, where Sonnet 4.5 retires on November 30.

Originally published on ImportStatic. This article was drafted with AI assistance; every claim was checked against the sources linked in it and the code samples were compiled and run.

How much MCP token overhead you pay depends on the harness

ImportStatic — Sat, 17 Oct 2026 06:21:35 +0000

Originally published on ImportStatic.

The issue-writing tool in GitHub's MCP server costs 807 tokens to describe to a model. The help page for gh issue create costs 869. So much for the idea that an MCP tool is inherently heavier than a CLI command.

That comparison comes from a small Go program we wrote after reading a Hacker News discussion of a post about an agent harness adding MCP support. One side said MCP eats context on every turn whether you use the tools or not. The other said it depends on the harness. That's an argument a measurement can settle. Both sides turn out to be right, and the MCP token overhead is easy to measure: GitHub's server with its default settings is 11,207 tokens of tool definitions, and whether those tokens reach the model is now a harness setting.

What we counted, and with which tokenizer

The program starts a server over stdio with the official Go SDK, lists its tools, and serializes each one the way a model API wants it: name, description and input schema. Then it counts tokens.

tiktoken.SetBpeLoader(tiktoken_loader.NewOfflineLoader())
enc, err := tiktoken.GetEncoding("o200k_base")
if err != nil {
    log.Fatal(err)
}
count := func(s string) int { return len(enc.Encode(s, nil, nil)) }

for tool, err := range session.Tools(ctx, nil) {
    if err != nil {
        log.Fatal(err)
    }
    def := functionTool{Name: tool.Name, Description: tool.Description, InputSchema: tool.InputSchema}
    defs = append(defs, def)
    r := row{
        name:   tool.Name,
        total:  countJSON(def),
        desc:   count(tool.Description),
        schema: countJSON(tool.InputSchema),
    }

Two caveats before the numbers. The tokenizer is o200k_base, which OpenAI's tiktoken table maps to the gpt-4o, gpt-4.1, gpt-5 and o-series names. Claude models have their own tokenizer, and the way to count with it is Anthropic's token counting endpoint. It's free and accepts a tools array, but it needs an API key and this machine doesn't have one. So these are OpenAI-encoding counts of compact JSON. Second, a provider is free to render tool definitions into the prompt in its own format, so what the model sees can differ from the JSON you send. Treat the figures as close, not exact.

All four servers were built from source at their latest tagged versions as of October 6, 2026, and started with dummy credentials. tools/list doesn't need real ones.

The MCP token overhead of four real servers

Server	Tools	Definitions	Per tool
gopls v0.23.0 (`gopls mcp`)	8	878 tokens	109
terraform-mcp-server v1.3.0	9	1,718 tokens	190
github-mcp-server v1.14.0, defaults	46	11,207 tokens	243
mcp-grafana v1.6.3	81	20,493 tokens	253
github-mcp-server v1.14.0, `--toolsets all`	91	23,206 tokens	255

Small servers are cheap. gopls gives a model eight tools for less than a page of text. The trouble starts when a server mirrors a whole product API: GitHub with every toolset plus Grafana is 43,699 tokens before the user has typed anything.

That lines up with what Anthropic says. Its tool search documentation gives the example of a five-server setup (GitHub, Slack, Sentry, Grafana and Splunk) that "can consume ~55k tokens in definitions before Claude does any work." Different tokenizer, different server versions, same order of magnitude.

And yes, without deferral you pay it on every request. Model APIs are stateless. Anthropic's pricing notes say tool use is billed on input tokens "including in the tools parameter", and that the parameter's tokens are the "tool names, descriptions, and schemas". On top of that sits a fixed tool-use system prompt, which Anthropic lists as 286 tokens for Claude Opus 5.5 and Sonnet 5.5. Next to 11,207, that part is noise.

Most of it is JSON Schema, not prose

Here's the full output for the GitHub server with default settings:

server            github-mcp-server v1.14.0
tools             46
definitions       11207 tokens (243 per tool)
  descriptions    1368 tokens
  input schemas   9287 tokens
output schemas    0 tokens on 0 tools (not included above)
annotations       964 tokens (not included above)
instructions      377 tokens
names only        185 tokens
names + 1st line  1190 tokens
largest tools:
  issue_write                                807 tokens (description 14, schema 783)
  pull_request_read                          560 tokens (description 12, schema 537)
  list_issues                                549 tokens (description 33, schema 505)
smallest tool:
  get_me                                      54 tokens (description 35, schema 9)

The tool descriptions, the part people imagine when they say "tool descriptions waste tokens", are 12% of the total. The input schemas are 83%.

Look at issue_write. Its description is one sentence, 14 tokens. Its schema is 783, because it has seventeen properties and several of them carry a paragraph of usage rules. This is one property out of the seventeen, copied from the server's response:

"parent_owner": {
  "description": "Repository owner of the parent issue. Must be provided with parent_repo. Omit both to use owner and repo. Only used when method is 'create' and parent_issue_number is provided.",
  "type": "string"
}

Grafana's server leans the other way, with 5,777 tokens of descriptions against 13,601 of schema, but schemas still dominate.

If you write a server, that's where to look. Merging "create" and "update" into one tool with a method argument, as GitHub does here, saves a tool slot and then spends the saving on parameters that only apply to one of the two methods. I wouldn't call that wrong. It's a trade, and it's worth knowing which side of it your tokens are on.

One more line in that output deserves a look: instructions. Servers can return free-text instructions when a client connects. GitHub's are 377 tokens with default toolsets and 1,559 with all of them. Terraform's are 924, more than half the size of its nine tool definitions. If you audit a server, count those too.

A CLI isn't smaller, it's just lazy

The usual counter-proposal is to skip MCP and let the agent run gh. We built gh v2.102.0 from source and counted its help output with the same tokenizer.

What the agent reads	Tokens
`gh --help`	610
`gh issue --help`	346
`gh issue create --help`	869
`gh issue edit --help`	1,068
MCP `issue_write` (create and update)	807

An agent that starts cold and walks down to gh issue create reads 1,825 tokens of help. The two help pages that cover what issue_write does add up to 1,937. Operation for operation, the MCP definition is the more compact of the two.

What the CLI has going for it is timing. Nothing is loaded until the agent asks, and an agent that never files an issue never reads the page. A model may also know gh well enough from training to skip the help entirely; we can't measure that here, and it won't be true for your internal tools.

So the real difference was never the format. It's eager versus lazy loading. And that is exactly the part harnesses have changed.

With tool search, the bill drops to a list of names

Anthropic's API has a tool search tool. You mark tools with defer_loading: true, and the docs are specific about what that does: it "controls what enters the context window, not what you send in the request." The definitions still travel in the tools array, but the model only sees the ones a search returns, up to five per search by default. Anthropic reports this "typically reduces" definition tokens "by over 85 percent". For MCP servers attached through its connector, you set the flag once for the whole server.

OpenAI's Responses API has the same idea: a tool_search tool, supported on gpt-5.4 and later, plus defer_loading on functions or MCP servers. For a deferred server the model initially sees, in the guide's words, "only the namespace or server name and description". OpenAI recommends keeping each namespace under ten functions.

Claude Code turns tool search on by default. Its docs list the setups that go without: a custom ANTHROPIC_BASE_URL, ENABLE_TOOL_SEARCH=false, and older models on Google Cloud. With it on, the docs describe Claude Code listing a server's tool names to the model, which then goes through a ToolSearch call to use them.

How small is a list of names? For GitHub's 46 default tools our program counted 185 tokens, against 11,207 for the definitions. Each harness formats its own list, so that's a floor and not a prediction. Then add what a search pulls in: five average GitHub tools is about 1,200 tokens, and only on the turns that need them.

That's the answer to the thread. In a harness that defers, MCP's standing cost is a few hundred tokens. In one that doesn't, it's the whole table above on every call.

What to do if your harness loads everything

Find out first. If you call a model API yourself and pass the tools from tools/list straight through, you're loading everything. (The stateless revision of the spec changes how a client fetches that list, not how big it is.)

Then trim at the server. GitHub's README says it plainly: "Enabling only the toolsets that you need can help the LLM with tool choice and reduce the context size." We measured what the flags buy:

github-mcp-server v1.14.0 flags	Tools	Definitions
`--toolsets all`	91	23,206 tokens
(defaults)	46	11,207 tokens
`--read-only`	27	6,492 tokens
`--toolsets issues`	10	3,281 tokens
`--tools issue_read,issue_write,list_issues`	3	1,693 tokens

An agent that triages issues doesn't need 91 tools, and a server started with --tools costs it 7% of the full set.

If you can use tool search, use it, and keep the handful of tools you call constantly non-deferred. Anthropic suggests three to five. Its own threshold for bothering is ten or more tools, or more than 10k tokens of definitions, which the GitHub server crosses on defaults.

And measure your own setup instead of trusting anyone's table, including ours. With an Anthropic key, send your real tools array to count_tokens for the model you run; the docs note that Claude Opus 4.7 and later produce roughly 30 percent more tokens for the same text than earlier Claude models did, so an old count is stale. Without a key, a short Go program and a published tokenizer will tell you which three tools are eating your context.

Originally published on ImportStatic. This article was drafted with AI assistance; every claim was checked against the sources linked in it and the code samples were compiled and run.

Go 1.27's JSON v2 is already under your encoding/json

ImportStatic — Mon, 12 Oct 2026 02:12:26 +0000

Originally published on ImportStatic.

Go 1.27 put encoding/json/v2 in the standard library, and the part most people will miss is that you are already running it. Since Go 1.27, the old encoding/json package is implemented on top of the v2 engine with a set of compatibility options switched on. Your output should not change, but your error strings can, and the moment you change an import to encoding/json/v2 the defaults flip: nil slices become [], field names match case-sensitively, duplicate keys are rejected and time.Duration stops marshaling at all.

What Go 1.27 changed even if you never import v2

The Go 1.27 release notes describe two new packages and one swapped engine:

encoding/json/v2 is the new semantic API: Marshal, Unmarshal, MarshalWrite, UnmarshalRead, MarshalEncode and UnmarshalDecode, each taking variadic Options.
encoding/json/jsontext is the lower-level syntactic layer, with an Encoder and Decoder that work on tokens and raw values.

The notes then say that encoding/json "is now backed by the v2 implementation. Marshaling and unmarshaling behavior is preserved, but the exact text of error messages may differ." The v1 API stays supported and nobody is required to migrate. If something does break after the upgrade, building with GOEXPERIMENT=nojsonv2 restores the original code, but the release notes say that opt-out "is expected to be removed in a future release."

So the first migration step costs nothing: upgrade to 1.27 and run your tests. The one category to look for is tests that compare err.Error() against a fixed string. Those can fail without any behaviour change, and the fix is to assert on error types such as *json.SyntaxError or *json.UnmarshalTypeError instead.

Go JSON v2 vs v1, side by side

The package documentation for encoding/json has a "Migrating to v2" section listing every behaviour difference and the option that controls it. Reading a list is one thing; seeing your own structs change is another. Here is the struct we used:

type User struct {
    Name    string   `json:"name"`
    Tags    []string `json:"tags"`
    Admin   bool     `json:"admin,omitempty"`
    Retries int      `json:"retries,omitempty"`
}

The program marshals and unmarshals the same values with jsonv1 "encoding/json" and "encoding/json/v2" and prints both. This is the real output:

== nil slice and omitempty on bool/int ==
v1   {"name":"ana","tags":null}
v2   {"name":"ana","tags":[],"admin":false,"retries":0}
== case-insensitive field names ==
v1   "bob" err=<nil>
v2   "" err=<nil>
== duplicate names ==
v1   "mallory" err=<nil>
v2   "alice" err=jsontext: duplicate object member name "name"
== invalid UTF-8 ==
v1   "caf�" err=<nil>
v2   "" err=jsontext: invalid UTF-8 within "/name" after offset 12
== time.Duration ==
v1   {"name":"backup","timeout":90000000000}
v2   error: json: unable to marshal from Go time.Duration within "/timeout": no default representation
v2+  {"name":"backup","timeout":90000000000}
== HTML escaping ==
v1   {"q":"<a&b>"}
v2   {"q":"<a&b>"}
v2+  {"q":"<a&b>"}

Going through them in order of how likely they are to hurt.

Nil slices and maps. v1 writes null for a nil slice or map; v2 writes [] and {}. That is friendlier to JavaScript clients and a silent contract change for anyone who checks === null. json.FormatNilSliceAsNull(true) and json.FormatNilMapAsNull(true) bring the old output back.

omitempty means something else. The first line shows "admin":false,"retries":0 reappearing. In v1, omitempty drops false, 0, nil pointers and nil interfaces. In v2 it drops a field only if it would encode as JSON null, "", {} or []. The documentation says the two agree for strings, slices, maps and arrays, and that existing uses on a bool, number, pointer or interface "should migrate to specifying omitzero instead", which works the same in both versions.

Case-insensitive field matching is gone. v1 happily put {"NAME":"bob"} into the Name field. v2 matches names exactly, so the field stays empty and, notice, no error is returned. If your clients send inconsistent casing, you will find out from missing data, not from logs. json.MatchCaseInsensitiveNames(true) restores the loose match for a whole call; the case:ignore tag option does it per field.

Duplicate keys are an error. v1 let the last "name" win, which is how {"name":"alice","name":"mallory"} became mallory. The v2 documentation's security section explains why this matters when two services parse the same request differently. v2 rejects it unless you pass jsontext.AllowDuplicateNames(true). One detail from our run: after the error, the destination already held "alice". Treat the struct as garbage once Unmarshal returns an error.

Invalid UTF-8 is an error. v1 replaced the bad byte with U+FFFD; v2 refuses, unless jsontext.AllowInvalidUTF8(true) is set. If you ingest data from old Latin-1 systems, this is the one that will show up in production.

time.Duration has no representation. In v2, a time.Duration "has no default representation and results in a SemanticError". That is the v2 error line. The v2+ line passes jsonv1.FormatDurationAsNano(true), an option that lives in the v1 package, and gets the old nanosecond integer back.

HTML escaping is off. v2 uses minimal escaping, so <a&b> goes out as is. If you embed JSON in HTML <script> blocks, keep jsontext.EscapeForHTML(true).

The documentation lists a few more that our structs did not hit: Go arrays must now be unmarshaled from a JSON array of exactly the same length, byte arrays (not slices) become Base64 strings, maps are no longer sorted when marshaled unless you pass json.Deterministic(true), the string tag option only applies to values that encode as numbers, unmarshaling null always zeroes the target, and malformed struct tags are reported as errors at runtime instead of being ignored.

Tags that give the same JSON under v1 and v2

You do not have to pick a side per call site. Most differences can be pinned on the type, so the same struct encodes identically whichever package touches it. This version of the struct passes a test that marshals with both and compares bytes:

// Portable keeps the same JSON under v1 and v2.
type Portable struct {
    Name    string   `json:"name,case:ignore"`
    Tags    []string `json:"tags,omitempty"`
    Admin   bool     `json:"admin,omitzero"`
    Retries int      `json:"retries,omitzero"`
}

func TestPortableSameBytes(t *testing.T) {
    for _, p := range []Portable{{Name: "ana"}, {Name: "bo", Tags: []string{"x"}, Admin: true, Retries: 3}} {
        b1, err := jsonv1.Marshal(p)
        if err != nil {
            t.Fatal(err)
        }
        b2, err := json.Marshal(p)
        if err != nil {
            t.Fatal(err)
        }
        if string(b1) != string(b2) {
            t.Errorf("v1 %s != v2 %s", b1, b2)
        }
        t.Logf("%s", b2)
    }
}

=== RUN   TestPortableSameBytes
    main_test.go:31: {"name":"ana"}
    main_test.go:31: {"name":"bo","tags":["x"],"admin":true,"retries":3}
--- PASS: TestPortableSameBytes (0.00s)

omitempty on the slice makes nil and empty both disappear, which sidesteps the null versus [] question. omitzero replaces omitempty on the bool and int. case:ignore keeps accepting "NAME" under v2, and a second test confirmed it decodes into Name.

While you are in there, v2 has an option v1 never had. json.RejectUnknownMembers(true) turns a typo or an unexpected field into an error that wraps json.ErrUnknownName:

json: cannot unmarshal JSON string into Go main.Portable: unknown object member name "role"

That is worth having on any request body that reaches an authorization check.

If you used v2 under GOEXPERIMENT=jsonv2 on Go 1.25 or 1.26, check your tags again. The release notes list changes made before the package became final: the format and unknown tag options were removed, the DiscardUnknownMembers option and SkipFunc were removed, and the inline tag option was renamed embed. Blog posts written during the experiment still show format: tags that will not work on 1.27.

Migrating one option at a time with DefaultOptionsV1

For a large codebase, the safe path is the one the official migration guide calls option-by-option. Change call sites to the v2 functions but pass jsonv1.DefaultOptionsV1(), which the guide calls "a trivial and safe change". Then switch individual behaviours to v2 by adding options after it, since later options override earlier ones:

b, err = json.Marshal(u, jsonv1.DefaultOptionsV1())
show("v2v1", b, err)
b, err = json.Marshal(u, jsonv1.DefaultOptionsV1(), json.FormatNilSliceAsNull(false))
show("mix", b, err)

v2v1 {"name":"ana","tags":null}
mix  {"name":"ana","tags":[]}

The first call is byte-for-byte v1. The second is v1 except for nil slices. Each option you flip is a small, reviewable change, and when the list of overrides covers everything you can drop DefaultOptionsV1() entirely.

For servers where you cannot predict every payload from tests, the guide points to github.com/go-json-experiment/jsonsplit. It has call modes such as CallBothButReturnV1, which runs both implementations, returns the v1 result and reports any difference, so production traffic tells you which options you need before you switch to CallBothButReturnV2 or OnlyCallV2. The guide warns that running both "will approximately double the cost of marshaling". Use it for the migration window, not permanently.

What we measured: faster unmarshal, slower marshal

The release notes say "marshal performance is broadly at parity with the previous implementation, while unmarshal performance is significantly faster." We checked that on one payload: a 500-element array of a struct with an int64, a string, a float, a three-element string slice, a two-entry map[string]string and a nested struct, about 79 KB of JSON. The benchmark calls plain encoding/json, run twelve times on the default 1.27.1 build and six times with GOEXPERIMENT=nojsonv2, on an Apple M4 Pro:

encoding/json on Go 1.27.1	Unmarshal	Allocs	Marshal	Allocs
v2 engine (default)	~695 µs	5,011	~336 µs	2,003
old engine (`nojsonv2`)	~920 µs	7,018	~241 µs	2,502

Times are rounded medians; the unmarshal runs were noisy (537 to 878 µs on the new engine), while the allocation counts were identical on every run. Unmarshal came out about 25% faster with 29% fewer allocations, which matches the release notes. Marshal took about 39% longer on this payload, which does not match "broadly at parity". One payload on one machine is not a verdict, but it is enough to say: if marshaling is on your hot path, benchmark it on 1.27 with and without nojsonv2 before you roll out.

If you are upgrading for the other Go 1.27 changes as well, the new synctest.Sleep and in-memory test server are worth a look in the same pass.

Originally published on ImportStatic. This article was drafted with AI assistance; every claim was checked against the sources linked in it and the code samples were compiled and run.

Claude Haiku 5.5 pricing jumps fivefold at 100,001 prompt tokens

ImportStatic — Thu, 08 Oct 2026 01:03:42 +0000

A request to Claude Haiku 5.5 with a 100,000-token prompt and 2,000 tokens of output costs $0.011. Add one token to the prompt and the same request costs $0.055. Nothing else changed; you crossed a line in the price list.

Anthropic released Haiku 5.5 on October 7, 2026, and as of October 8 it's available on the Claude API, Amazon Bedrock, Google Cloud, Microsoft Foundry and Claude Platform on AWS under the ID claude-haiku-5-5 (anthropic.claude-haiku-5-5 on Bedrock). The headline is the price: $0.10 per million input tokens against $1 for Haiku 4.5. The catch is that the cheap rate is for "prompts up to 100,000 tokens", and the model takes prompts ten times that size. A Hacker News discussion of the release went back and forth on whether agent workloads can stay under the line and didn't settle it. So we did the arithmetic with a small Go program. We had no API key in this run, so nothing below was sent to the model: the prices come from Anthropic's documentation and the costs are computed from them.

Claude Haiku 5.5 pricing is two complete price lists

The pricing page gives Haiku 5.5 two rows, one "for prompts up to 100,000 tokens" and one "for prompts over 100,000 tokens". Per million tokens:

	Up to 100,000	Over 100,000	Haiku 4.5
Input	$0.10	$0.50	$1.00
Output	$0.50	$2.50	$5.00
Cache read	$0.01	$0.05	$0.10
Cache write, 5 minutes	$0.125	$0.625	$1.25
Cache write, 1 hour	$0.20	$1.00	$2.00
Batch input / output	$0.05 / $0.25	$0.25 / $1.25	$0.50 / $2.50

Every rate in the second column is five times the first. That includes output, which is the part people miss: the length of what you send decides the price of what comes back.

This isn't a marginal bracket like income tax, where only the excess pays the higher rate. The row is chosen per prompt and applies to the whole request. It's also unique in the current lineup. For the other 1M-context models the same page says "A 900k-token request is billed at the same per-token rate as a 9k-token request", and it names Haiku 5.5 as the exception.

What the docs say counts as the prompt, and what they leave out

"Prompt" is doing a lot of work in that table, and the documentation doesn't define it as a formula over the usage fields. Here's what it does say.

With prompt caching on, the input is split across input_tokens, cache_creation_input_tokens and cache_read_input_tokens. The context windows page is blunt about cached tokens: "prompt caching changes what you pay for those tokens, not whether they count." So we treat the prompt as the sum of all three, which is the reading where a 100,000-token cached prefix plus a 50-token question lands in the expensive tier. That's our reading, not a quoted rule. If your bill depends on it, check a real invoice against a request you know the size of.

Two more things move the line.

Haiku 5.5 uses the newer tokenizer, and the migration guide says the same text "produces approximately 30% more tokens" than on Haiku 4.5. If you know your prompt sizes in Haiku 4.5 tokens, the threshold isn't 100,000 for you. It's about 76,900.

And the pre-flight check is approximate. The token counting endpoint is free and takes the same body as a message request, but the docs call its result an estimate that "might differ by a small amount". Don't route on <= 100000. Leave headroom.

curl https://api.anthropic.com/v1/messages/count_tokens \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "content-type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model": "claude-haiku-5-5", "messages": [{"role": "user", "content": "..."}]}'

That's the documented request with the model swapped in; we didn't send it. Pass claude-haiku-5-5 and not your old model, or you'll get the old tokenizer's count.

One token over, priced in Go

The calculator is one function. It mirrors the usage object, picks a tier from the prompt length and applies that tier to everything:

type Usage struct {
    InputTokens              int `json:"input_tokens"`
    CacheCreationInputTokens int `json:"cache_creation_input_tokens"`
    CacheReadInputTokens     int `json:"cache_read_input_tokens"`
    OutputTokens             int `json:"output_tokens"`
}

// Rates are US dollars per million tokens.
type Rates struct {
    Input, CacheWrite5m, CacheRead, Output float64
}

var (
    Haiku55Short = Rates{Input: 0.10, CacheWrite5m: 0.125, CacheRead: 0.01, Output: 0.50}
    Haiku55Long  = Rates{Input: 0.50, CacheWrite5m: 0.625, CacheRead: 0.05, Output: 2.50}
    Haiku45      = Rates{Input: 1.00, CacheWrite5m: 1.25, CacheRead: 0.10, Output: 5.00}
)

const Haiku55Threshold = 100_000

func (u Usage) PromptTokens() int {
    return u.InputTokens + u.CacheCreationInputTokens + u.CacheReadInputTokens
}

func (u Usage) at(r Rates) float64 {
    return (float64(u.InputTokens)*r.Input +
        float64(u.CacheCreationInputTokens)*r.CacheWrite5m +
        float64(u.CacheReadInputTokens)*r.CacheRead +
        float64(u.OutputTokens)*r.Output) / 1e6
}

func Haiku55Cost(u Usage) (dollars float64, long bool) {
    if u.PromptTokens() > Haiku55Threshold {
        return u.at(Haiku55Long), true
    }
    return u.at(Haiku55Short), false
}

Feed it two requests that differ by one token, then one 150,000-token prompt against the same material in two halves (1,000 output tokens each):

prompt  100000 + 2000 out  $0.011000  long=false
prompt  100001 + 2000 out  $0.055001  long=true

one request  $0.0775
two requests $0.0160

The second pair is the practical one. Summarising a long document in two 75,000-token calls costs a fifth of doing it in one, and you can add a third call to merge the halves and still be far ahead. For extraction and classification over big inputs, which is what this model is sold for, chunking just became a pricing decision as well as a quality one.

An agent loop crosses the line on turn 17

Is 100,000 tokens a lot? For a single classification call, it's enormous. For a tool-calling agent that appends every result to the conversation, it isn't.

We modelled a loop with prompt caching on: an 8,000-token prefix of system prompt and tool definitions, and each turn adding a 5,000-token tool result and a 500-token reply. Those sizes are made up to be plausible. This is a model of the billing, and your agent will have different numbers.

turn 16: prompt  95500  $0.00184  long=false
turn 17: prompt 101000  $0.00946  long=true
append only          $0.3267  (first long-tier turn: 17)
compact before 100k  $0.0623  (first long-tier turn: 0, compactions: 2)

Turn 17 costs five times turn 16 for almost the same work, and every later turn stays in the expensive tier because the history only grows. Over 40 turns the append-only loop costs $0.33. The second line replaces the history with a 3,000-token summary whenever the next prompt would cross 100,000, pays for writing that summary, and comes to $0.06. Whether a summary is good enough for your task is a separate question that no price table answers.

There's a trap in the obvious fix. The API's threshold compaction, in beta and listed as supporting Haiku 5.5, summarises old turns on the server once the input reaches a trigger. The default trigger is 150,000 input tokens. On every other model that's a sensible default. On this one it means compaction switches on 50,000 tokens after the price went up. Set it yourself; the minimum is 50,000:

{
  "model": "claude-haiku-5-5",
  "max_tokens": 4096,
  "context_management": {
    "edits": [
      { "type": "compact_20260112", "trigger": { "type": "input_tokens", "value": 80000 } }
    ]
  },
  "messages": [{ "role": "user", "content": "..." }]
}

That body is adapted from the documented example and goes with the anthropic-beta: compact-2026-01-12 header. We didn't send it either.

Don't trim the history by hand without reading the migration guide first. Haiku 5.5 thinks by default, and a request that sends a thinking block back after earlier messages changed can come back as a 400 error (the guide says which accounts get the check). Its instruction is to keep conversations append-only, which is exactly what a hand-rolled "drop the oldest tool results" loop doesn't do.

It's still cheaper than Haiku 4.5 on both sides

None of this makes Haiku 5.5 a bad deal. The long tier is half of Haiku 4.5's per-token price, and a quarter of Sonnet 5.5's $2 input rate.

The tokenizer eats into that, though. Pricing the same text on both models, with Haiku 5.5's counts inflated by the documented 30%:

threshold in Haiku 4.5 tokens: 76923
4.5:  10000 in  $0.0150 | 5.5:  13000 in  $0.0019 long=false | saving 87%
4.5:  50000 in  $0.0550 | 5.5:  65000 in  $0.0072 long=false | saving 87%
4.5:  76000 in  $0.0810 | 5.5:  98800 in  $0.0105 long=false | saving 87%
4.5:  78000 in  $0.0830 | 5.5: 101400 in  $0.0539 long=true  | saving 35%
4.5: 150000 in  $0.1550 | 5.5: 195000 in  $0.1008 long=true  | saving 35%

So: 87% cheaper for the same text under the line, 35% cheaper above it. Anthropic's announcement says prompts up to 100,000 tokens "make up around 90% of requests to our previous Haiku model". That's a share of requests, and long requests carry more tokens each, so the share of your spend that lands in the long tier can be a good deal higher than one in ten.

The switch isn't only a price change. Manual budget_tokens thinking, non-default temperature, and assistant prefill all return 400 errors on Haiku 5.5, much like the changes we listed for Claude Sonnet 5.5. And if your long prompts are long because of tool definitions, measure those first; they're in every request.

Our advice is short. Put a budget of 90,000 counted tokens on anything you send to Haiku 5.5, set the compaction trigger below it, and log usage per request so you can see how many calls go over. If a workload can't fit, split it. If it can't be split, run it anyway, and know that it's the 35% discount you're getting.