Performance Benefits of TOON Over JSON for LLM Prompts

TOON (Token-Oriented Object Notation) is a compact serialization format designed specifically for sending structured JSON data to large language models — think RAG pipelines, agent tool calls, and chatbot context windows — while staying human-readable. According to TOON's published benchmarks (see the TOON specification), TOON uses roughly 42.6% fewer tokens than equivalent JSON for typical structured data, with comparable-or-better LLM retrieval accuracy (72.2% vs JSON's 71.4% in their test suite). In this article, we'll unpack where those savings come from and why they matter for anyone paying for LLM prompt tokens or working within a context-window budget.

📊 Token & Size Comparison

The headline figure from TOON's published benchmarks is a ~42.6% reduction in tokens compared to equivalent JSON, measured across typical structured data sent to LLMs. The table below illustrates the kind of data shapes where that reduction shows up most — lists of records with the same fields, which is the common shape for API responses and database query results:

The reduction is largest for uniform arrays of objects — the classic shape of API responses and database query results — because that's exactly the case TOON's core mechanism targets: a single header row listing field names once (items[N]{col1,col2}:), followed by comma-separated rows, instead of repeating each object's keys the way JSON does for every single record.

🔄 Conversion Example

Here's a small, concrete example — a typical API response listing user records — shown in both formats. We counted the raw characters for this specific example (not a formal benchmark): the JSON version is 240 bytes and the TOON version is 113 bytes, about a 53% reduction for this one snippet. Actual savings on your own data will vary with shape and size, which is why TOON's own published benchmarks (~42.6% fewer tokens on average) are the more reliable reference point:

{
  "users": [
    { "id": 1, "name": "Alice Johnson", "role": "admin", "active": true },
    { "id": 2, "name": "Bob Smith", "role": "editor", "active": true },
    { "id": 3, "name": "Carol Lee", "role": "viewer", "active": false }
  ]
}

users[3]{id,name,role,active}:
  1,Alice Johnson,admin,true
  2,Bob Smith,editor,true
  3,Carol Lee,viewer,false

Notice what's happening: JSON repeats the keys id, name, role, and active for every single record. TOON declares the field names exactly once in the header ({id,name,role,active}) and then lists each row as plain comma-separated values. For a 3-row example the win is modest; for a 100-row API response, that repeated-key overhead in JSON compounds fast — which is exactly why this pattern is the single biggest lever behind TOON's token savings. TOON also eliminates:

• Quote marks around property names and simple string values
• Repeated keys for every object in a uniform array
• Curly braces for object delimiters
• Array brackets for simple arrays

🚀 Real-World Impact for LLM Prompts

These token savings matter most wherever structured data is sent into an LLM's context window — RAG retrieval, agent tool-call results, and chatbot prompts:

📈 Where These Numbers Come From

We haven't run our own large-scale benchmark study on this site. The token and accuracy figures cited above are the ones published by the TOON specification itself:

• Source: TOON's published benchmarks, part of the TOON specification and reference implementation
• Headline metric: ~42.6% fewer tokens than equivalent JSON for typical structured data
• Accuracy metric: 72.2% LLM retrieval accuracy for TOON vs 71.4% for JSON, in their test suite
• What we added: the small, illustrative JSON-vs-TOON example above, with its character counts computed directly from that one snippet

User-list example above:
JSON size:   240 bytes
TOON size:   113 bytes
Reduction:   ~52.9%

TOON's published benchmark (typical structured data):
Token reduction:      ~42.6% fewer tokens than JSON
Retrieval accuracy:   72.2% (TOON) vs 71.4% (JSON)

🌍 Where This Matters in LLM Workflows

TOON's token savings apply wherever structured data gets serialized into a prompt. A few common places that shows up:

💰 Cost Implications

Fewer tokens per prompt translate into real, direct savings for anyone calling an LLM API:

🔮 Where TOON Fits Going Forward

As LLM usage keeps shifting toward agentic workflows and larger retrieved context, a few practical considerations are worth keeping in mind:

💡 Recommendations

Based on how TOON's savings mechanism works, here are our recommendations for when to use it:

🚀 Ready to Cut Your Prompt Tokens?

Ready to see the token savings on your own data? Start by converting your existing JSON payloads and comparing the token count before and after: