Novita

OpenAI chat/completions · serves google/gemma-4-31b-it as gemma-4-31b · probed live 2026-07-27 · back to the matrix

Harness

C−

A − 2 red − 1 yellow, per the ladder; flags below.

Billing

✓

Reported usage consistent with bytes on the wire.

Caching

0/8 cold→warm trials hit — every call re-reads (and re-bills) the full context. Speed numbers →

Faithfulness

—

Not yet run on this provider.

Flags

red harness Images in tool results: rejected, 500'd, or silently blinded on 7 of 11 providers
The model reads images returned by tools — the reference provider delivers them, perception-judged, and three OpenAI-compat providers prove the standard shape works. Seven providers fail on schema choice, not model...
red harness Novita silently ignores tool_choice
"required" and forced-specific constraints are accepted and have no effect on decoding; only "none" is honored. The application consequence matches DeepInfra's: workflows that depend on forced tool calls silently receive...
yellow harness Tool results replayed out of order are mis-paired on most providers
The chat dialect's contract is that a tool message is matched to its call by tool_call_id, in any order. After two parallel calls, replaying the results reversed swaps the data between them on these providers — the...

Every flag links to its finding — evidence, repro, and disclosure records live there; the deepest findings have full writeup pages.