SambaNova bills identical requests inflated token counts

red · billing · probed live 2026-07-26 · affects SambaNova · run the repro yourself

The same request body, sent repeatedly, is not billed the same number of prompt tokens — and the errors overwhelmingly point up. Across three independent 40-call runs, SambaNova charged 9,176 tokens for 4,200 actually sent: 2.18× on average. The inflated figure is carried in total_tokens — the field an invoice sees — so this is what a customer pays, not a cosmetic reporting glitch.

What was measured

One request body — a 35-token prompt, byte-identical on every send — was sent 40 times in sequence, three separate times on 2026-07-26. A correct meter bills identical bytes identically, every time. Instead:

13 of the 120 calls (11%) were billed a wrong value; ten distinct values appeared, from 27 to 2,478 tokens, for one unchanging 35-token request.

The money is extra, not shuffled

An inflated count could in principle be another request's count, swapped — annoying but zero-sum. It is not. Proof by a closed pair: two requests that are both ours, fired together, so every token entering and leaving the set is known. A 74-token request co-fired with a 1,197-token request was billed 1,197 on both — the small request took the large one's count while the large one kept its own. The pair was billed 2,394 tokens for 1,271 sent: an excess of 1,123 tokens that nobody sent. Totals do not conserve; the overcharge is genuine extra billing.

What it is not

• Not hidden added text. The meter occasionally bills below the true count (27 for the 35-token prompt, seen in all three runs) — text a provider adds can only add tokens. Separately, a forced verbatim-echo test returned only what we sent, in order, nothing before it, and the token arithmetic leaves no budget for unsent text.
• Not context leaking between requests. A canary marker planted in one request never appeared in a co-fired one (0/3), and a secret-code question answerable only from the co-fired request came back empty-handed 3/3 — while both members were billed an identical 1,197. Requests share a billing statistic, not a context.

Mechanism — our working hypothesis

Cross-batch attribution: SambaNova serves requests in batches, and one batch member's token count gets stamped onto another member's bill. Supporting evidence: (a) the wrong value can be chosen in advance — co-fire a small request with a large one of known size and the small one is billed the large one's exact count; (b) co-fired responses return bit-identical time_to_first_token and end_time — direct proof that batch-level values are stamped onto every member; (c) the occasional under-bill is what being co-batched with a smaller neighbor looks like; (d) the wrong values in sequential runs (27…2,478) look like a sample of other customers' prompt sizes. The limit of the claim: foreign traffic is unobservable, so for sequential runs the neighbor's identity is inferred from consistency with the controlled co-fire experiment, not directly observed.

Why it matters

Spend on this provider cannot be predicted, reconciled, or attributed. A cost-control layer that meters against reported usage will mis-bill by multiples — and because the wrong values track other traffic, the error grows with how busy the endpoint is, so it is largest exactly when spend is highest.

Run it yourself

The repro is a standalone script with no dependencies beyond Node: one loop that sends the same request 40 times and prints what each call was billed. src/billing/sambanova-stable-overcharge-repro.ts in the suite — SAMBANOVA_API_KEY=<key> npx tsx src/billing/sambanova-stable-overcharge-repro.ts. A captured run:

call 10/40: billed prompt_tokens = 35   (total_tokens = 37)
call 11/40: billed prompt_tokens = 647  (total_tokens = 649)
call 12/40: billed prompt_tokens = 35   (total_tokens = 37)
…
call 22/40: billed prompt_tokens = 27   (total_tokens = 29)
…
call 37/40: billed prompt_tokens = 2478 (total_tokens = 2480)
------------------------------------------------------------
the same request was sent 40 times
distinct billed values: 27, 35, 647, 2478 (a correct meter shows exactly one)
the request's real size: 35 tokens
calls billed a wrong value: 4 of 40
tokens billed in total: 4439 — tokens actually sent: 1400
overcharge: 3.17x

Provider response

Have a response or a correction? joel@custlabs.com — it becomes part of this record.