AWS

2 graded endpoints serving gemma-4-31b · back to the matrix

A vendor is graded once per API it speaks, because a shim and a native protocol routinely behave differently — AWS speaks 2.

Billing

Footnote (customer-favoring, not flagged): Reports reasoning_tokens: 0 beside delivered reasoning — under-counts in the customer's favor.

Findings affecting AWS

• red harness Four providers drop replayed reasoning: the model never sees its own prior thinking
  The gemma-4 chat template renders reasoning replayed on model turns, and the model card mandates preserving thinking across tool-call turns — the reference provider reproduces this live. These four providers delete it...
• red harness Images in tool results: rejected, 500'd, or silently blinded on 7 of 11 providers
  The model reads images returned by tools — the reference provider delivers them, perception-judged, and three OpenAI-compat providers prove the standard shape works. Seven providers fail on schema choice, not model...
• yellow harness AWS Bedrock never emits parallel tool calls
  parallel_tool_calls: true is accepted on both Bedrock APIs, but only a single call ever comes back; every other provider emits two for the same two-lookup prompt. A model capability suppressed at the serving layer; the...
• yellow harness AWS Bedrock un-escapes tool-call arguments in transit
  At temperature 0, nine providers agree the model returns a tricky string argument with literal escape sequences. Both Bedrock APIs instead return real control characters — one extra unescape pass applied to the model's...
• yellow harness AWS Bedrock (Responses) rejects temperature
  temperature returns 400 unsupported_parameter, so deterministic sampling is unreachable on this provider: every run is stochastic, degrading reproducibility for applications and evals alike.