Google (native)reference

generateContent — Google's own protocol · serves gemma-4-31b-it as gemma-4-31b · probed live 2026-07-26 · back to the matrix

This is Google's native API. Google's OpenAI-compat shim is graded separately as its own provider: google-openai.

Harness

A−

A − 0 red − 1 yellow, per the ladder; flags below.

Billing

✓

Reported usage consistent with bytes on the wire.

Caching

testing

Context reuse across tool-call turns is still being probed on this provider.

Faithfulness

reference

This provider is the reference implementation the battery compares against.

Flags

yellow harness Tool results replayed out of order are mis-paired on most providers
The chat dialect's contract is that a tool message is matched to its call by tool_call_id, in any order. After two parallel calls, replaying the results reversed swaps the data between them on these providers — the...

Every flag links to its finding — evidence, repro, and disclosure records live there; the deepest findings have full writeup pages.