# SambaNova bills identical requests inflated token counts red · billing · probed live 2026-07-26 · affects SambaNova · HTML version: [https://inferencecanary.com/sambanova-overbilling/](https://inferencecanary.com/sambanova-overbilling/) The same request body, sent repeatedly, is not billed the same number of prompt tokens — and the errors overwhelmingly point up. Across three independent 40-call runs, SambaNova charged **9,176 tokens for 4,200 actually sent: 2.18× on average**. The inflated figure is carried in `total_tokens` — the field an invoice sees — so this is what a customer pays, not a cosmetic reporting glitch. ## What was measured One request body — a 35-token prompt, byte-identical on every send — was sent 40 times in sequence, three separate times on 2026-07-26. A correct meter bills identical bytes identically, every time. Instead: | Run | Billed for 1,400 sent | Overcharge | Wrong values seen | |---|---|---|---| | 1 | 2,810 | 2.01× | 27, 67, 221, 256, 459, 590 | | 2 | 1,927 | 1.38× | 27, 578 | | 3 | 4,439 | 3.17× | 27, 647, 2478 | 13 of the 120 calls (11%) were billed a wrong value; ten distinct values appeared, from 27 to 2,478 tokens, for one unchanging 35-token request. ## The money is extra, not shuffled An inflated count could in principle be another request's count, swapped — annoying but zero-sum. It is not. Proof by a closed pair: two requests that are *both ours*, fired together, so every token entering and leaving the set is known. A 74-token request co-fired with a 1,197-token request was billed **1,197 on both** — the small request took the large one's count while the large one *kept its own*. The pair was billed 2,394 tokens for 1,271 sent: an excess of 1,123 tokens that nobody sent. Totals do not conserve; the overcharge is genuine extra billing. ## Mechanism — our working hypothesis **Cross-batch attribution**: SambaNova serves requests in batches, and one batch member's token count gets stamped onto another member's bill. Supporting evidence: (a) the wrong value can be chosen in advance — co-fire a small request with a large one of known size and the small one is billed the large one's *exact* count; (b) co-fired responses return bit-identical `time_to_first_token` and `end_time` — direct proof that batch-level values are stamped onto every member; (c) the occasional under-bill is what being co-batched with a *smaller* neighbor looks like; (d) the wrong values in sequential runs (27…2,478) look like a sample of other customers' prompt sizes. The limit of the claim: foreign traffic is unobservable, so for sequential runs the neighbor's identity is inferred from consistency with the controlled co-fire experiment, not directly observed. ## Why it matters Spend on this provider cannot be predicted, reconciled, or attributed. A cost-control layer that meters against reported usage will mis-bill by multiples — and because the wrong values track other traffic, the error grows with how busy the endpoint is, so it is largest exactly when spend is highest. ## Run it yourself The repro is a standalone script with no dependencies beyond Node: one loop that sends the same request 40 times and prints what each call was billed. Save it as `sambanova-overcharge-repro.ts` and run `SAMBANOVA_API_KEY= npx tsx sambanova-overcharge-repro.ts`. ```typescript /** * SambaNova over-billing repro. * * Sends the exact same request N times, one after another, and prints the * number of prompt tokens SambaNova billed for each call. A correct meter * bills identical bytes identically, every time. Seeing more than one value * below proves the meter is broken; the totals at the end show the overcharge. * * Run: SAMBANOVA_API_KEY= npx tsx sambanova-overcharge-repro.ts */ const API_KEY = process.env.SAMBANOVA_API_KEY; if (!API_KEY) { console.error("Set SAMBANOVA_API_KEY first."); process.exit(1); } const N = 40; // This exact body is sent every time. Nothing about it ever changes. const REQUEST_BODY = JSON.stringify({ model: "gemma-4-31B-it", messages: [{ role: "user", content: "What is 6 times 7? Reply with just the number." }], temperature: 0, max_tokens: 5, chat_template_kwargs: { enable_thinking: false }, }); const billed: number[] = []; for (let i = 1; i <= N; i++) { const res = await fetch("https://api.sambanova.ai/v1/chat/completions", { method: "POST", headers: { Authorization: `Bearer ${API_KEY}`, "Content-Type": "application/json" }, body: REQUEST_BODY, }); if (!res.ok) throw new Error(`HTTP ${res.status}: ${await res.text()}`); const json = (await res.json()) as { usage: { prompt_tokens: number; total_tokens: number } }; billed.push(json.usage.prompt_tokens); console.log(`call ${String(i).padStart(2)}/${N}: billed prompt_tokens = ${json.usage.prompt_tokens} (total_tokens = ${json.usage.total_tokens})`); await new Promise((r) => setTimeout(r, 1200)); } // The request's real size is whatever value the meter reports most often. const counts = new Map(); for (const b of billed) counts.set(b, (counts.get(b) ?? 0) + 1); const trueCount = [...counts.entries()].sort((a, b) => b[1] - a[1])[0]![0]; const distinct = [...counts.keys()].sort((a, b) => a - b); const wrongCalls = billed.filter((b) => b !== trueCount).length; const totalBilled = billed.reduce((a, b) => a + b, 0); const totalSent = trueCount * N; console.log("------------------------------------------------------------"); console.log(`the same request was sent ${N} times`); console.log(`distinct billed values: ${distinct.join(", ")} (a correct meter shows exactly one)`); console.log(`the request's real size: ${trueCount} tokens`); console.log(`calls billed a wrong value: ${wrongCalls} of ${N}`); console.log(`tokens billed in total: ${totalBilled} — tokens actually sent: ${totalSent}`); console.log(`overcharge: ${(totalBilled / totalSent).toFixed(2)}x`); ``` A captured run — the 3.17× one from the table above, complete: ``` call 1/40: billed prompt_tokens = 35 (total_tokens = 37) call 2/40: billed prompt_tokens = 35 (total_tokens = 37) call 3/40: billed prompt_tokens = 35 (total_tokens = 37) call 4/40: billed prompt_tokens = 35 (total_tokens = 37) call 5/40: billed prompt_tokens = 35 (total_tokens = 37) call 6/40: billed prompt_tokens = 35 (total_tokens = 37) call 7/40: billed prompt_tokens = 35 (total_tokens = 37) call 8/40: billed prompt_tokens = 35 (total_tokens = 37) call 9/40: billed prompt_tokens = 35 (total_tokens = 37) call 10/40: billed prompt_tokens = 35 (total_tokens = 37) call 11/40: billed prompt_tokens = 647 (total_tokens = 649) call 12/40: billed prompt_tokens = 35 (total_tokens = 37) call 13/40: billed prompt_tokens = 35 (total_tokens = 37) call 14/40: billed prompt_tokens = 35 (total_tokens = 37) call 15/40: billed prompt_tokens = 35 (total_tokens = 37) call 16/40: billed prompt_tokens = 35 (total_tokens = 37) call 17/40: billed prompt_tokens = 35 (total_tokens = 37) call 18/40: billed prompt_tokens = 35 (total_tokens = 37) call 19/40: billed prompt_tokens = 35 (total_tokens = 37) call 20/40: billed prompt_tokens = 35 (total_tokens = 37) call 21/40: billed prompt_tokens = 35 (total_tokens = 37) call 22/40: billed prompt_tokens = 27 (total_tokens = 29) call 23/40: billed prompt_tokens = 35 (total_tokens = 37) call 24/40: billed prompt_tokens = 35 (total_tokens = 37) call 25/40: billed prompt_tokens = 35 (total_tokens = 37) call 26/40: billed prompt_tokens = 27 (total_tokens = 29) call 27/40: billed prompt_tokens = 35 (total_tokens = 37) call 28/40: billed prompt_tokens = 35 (total_tokens = 37) call 29/40: billed prompt_tokens = 35 (total_tokens = 37) call 30/40: billed prompt_tokens = 35 (total_tokens = 37) call 31/40: billed prompt_tokens = 35 (total_tokens = 37) call 32/40: billed prompt_tokens = 35 (total_tokens = 37) call 33/40: billed prompt_tokens = 35 (total_tokens = 37) call 34/40: billed prompt_tokens = 35 (total_tokens = 37) call 35/40: billed prompt_tokens = 35 (total_tokens = 37) call 36/40: billed prompt_tokens = 35 (total_tokens = 37) call 37/40: billed prompt_tokens = 2478 (total_tokens = 2480) call 38/40: billed prompt_tokens = 35 (total_tokens = 37) call 39/40: billed prompt_tokens = 35 (total_tokens = 37) call 40/40: billed prompt_tokens = 35 (total_tokens = 37) ------------------------------------------------------------ the same request was sent 40 times distinct billed values: 27, 35, 647, 2478 (a correct meter shows exactly one) the request's real size: 35 tokens calls billed a wrong value: 4 of 40 tokens billed in total: 4439 — tokens actually sent: 1400 overcharge: 3.17x ``` ## Provider response > Reported to SambaNova (community thread): https://community.sambanova.ai/t/sambanova-2x-overcharge-for-tokens-identical-requests-billed-inflated-token-counts/1698/2 — The response did not address the defect. Reproducible as of 2026-07-26. Have a response or a correction? joel@custlabs.com — it becomes part of this record.