Your agent doesn't have a JSON problem. It has a decoding problem.

Most production agents I've reviewed ship some version of this pattern:

→ Call the LLM. → Try to json.loads the output. → On JSONDecodeError, re-prompt with "Respond ONLY in valid JSON" and call the LLM again.

It mostly works. ~5–10% of calls retry. Each retry is a second full inference - a tax on throughput, latency, and GPU cost for a problem unrelated to the model's reasoning.

The fix isn't a better prompt. It's grammar-constrained decoding (xgrammar in SGLang, Outlines in vLLM). The decoder masks every token that would produce invalid output. Result is always parseable, by construction.

The biggest payoff isn't on tool-call args. It's on the router. Define a Pydantic schema -

agent: Literal["billing", "support", "sales", "escalate"] confidence: float reason: str

hand it to the decoder, and the router cannot pick an agent that doesn't exist. Cannot malform confidence. Cannot omit reason. Structural correctness is guaranteed before the first token decodes.

PRODUCTION NUMBERS (Llama-3.3-70B router, H100, SGLang + xgrammar): • Parse-error rate: ~7% → 0% • Per-decode masking overhead: ~5–15ms • Net per-user tok/s: higher - no retry tokens

THE TRAP: over-constrained schemas make the model lie. Always include an "escalate" or null option in your enum, or nonsense queries get routed somewhere wrong with high confidence.

If your agent still wraps every model call in try/except JSONDecodeError on the hot path, you're paying for a problem that's been solved for two years.