Design Principles Behind the TOON Format

TOON (Token-Oriented Object Notation) exists to solve a specific, practical problem: sending structured JSON data to a large language model burns far more tokens than the data actually requires. Every object in a JSON array repeats its own keys, braces, and quotes add punctuation that carries no information for the model, and pretty-printed indentation multiplies whitespace. In prompts, RAG pipelines, agent tool calls, and anywhere else structured data has to fit inside a context window, those extra tokens cost money and eat into the space available for everything else. This article walks through the reasoning behind TOON's core design choices and why each one follows directly from that goal.

👁️ Principle 1: Minimize Repeated Tokens

The biggest source of waste in JSON arrays is repetition: every object in the array repeats the same set of field names. A 100-row array of uniform objects means the same keys get sent to the model 100 times. TOON's tabular array syntax states the field names exactly once, in a header, and then lists each row as plain comma-separated values underneath it.

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "editor" },
    { "id": 3, "name": "Carol", "role": "viewer" }
  ]
}

users[3]{id,name,role}:
    1,Alice,admin
    2,Bob,editor
    3,Carol,viewer

The [3] length marker and {id,name,role} header tell an LLM exactly how many rows to expect and what each column means, before it reads a single row. The field names appear once instead of three times, and the row data itself is plain comma-separated values with no repeated braces or quotes.

🤝 Principle 2: Human-Readable, Not Just Byte-Efficient

Token efficiency is easy to achieve if readability is thrown away entirely — minified JSON is already about as byte-efficient as JSON gets, but it is unpleasant for a person to scan or debug. Verbose, pretty-printed JSON is readable but reintroduces all the token overhead. TOON aims to sit between those two extremes: indentation-based structure that a person can scan the way they would scan YAML, without JSON's braces, quotes, and repeated-comma punctuation getting in the way.

🔄 Principle 3: Lossless Compatibility with JSON

A token-efficient format is only useful in practice if it can slot into pipelines that already speak JSON. TOON is designed to round-trip losslessly: any JSON document can be converted to TOON and back without losing or altering data. That means TOON only needs to sit at the LLM-prompt boundary — the rest of an application, its database, and its APIs can stay in JSON.

⚡ Principle 4: LLM-Parseable in Both Directions

TOON isn't only something a model reads — in many agent and tool-calling workflows, a model is also asked to generate structured output. That means the syntax has to be unambiguous enough for an LLM to produce correctly, not just for a human to read. Explicit array lengths and column headers give the model (and any validator downstream) a way to check that a generated block has the right shape before it's even parsed.

🏗️ Principle 5: A Narrow, Well-Defined Scope

TOON deliberately doesn't try to replace JSON everywhere. It targets one boundary: the point where structured data crosses into an LLM prompt, RAG context, or tool-call payload. Everywhere else — storage, APIs, application logic — JSON keeps doing its job. Because conversion between the two is lossless, TOON can be adopted at just that one boundary without requiring any other part of a system to change.

# Application code, database, APIs: stay in JSON
response = call_api()  # returns JSON

# At the LLM boundary: convert to TOON to save tokens
prompt_context = json_to_toon(response)
send_to_llm(prompt_context)

# If the model returns TOON, convert back losslessly
result = toon_to_json(llm_output)

Nothing upstream or downstream of the LLM call needs to know TOON exists — the conversion is isolated to the moment data enters or leaves the model.

🎨 Designing for AI-Facing Data

TOON is designed around the shapes of data that actually show up at the LLM boundary: retrieval results, database rows, API responses, and tool-call arguments. These are usually collections of uniform objects, which is exactly the case where JSON's per-item key repetition is most wasteful and where a tabular representation pays off the most.

🔬 Where These Principles Come From

These principles follow directly from how the format is structured, not from a separate research program. TOON's tabular array syntax mechanically removes repeated keys; its indentation-based layout mechanically removes repeated braces and quotes; and its lossless JSON round-trip is a property of the conversion logic itself. According to TOON's published benchmarks, applying these mechanics to typical structured datasets produces roughly 42.6% fewer tokens than the equivalent JSON, with retrieval accuracy in LLM evaluations that is comparable to or slightly better than JSON (72.2% versus JSON's 71.4%). The full methodology and benchmark data are published in the project's own repository, linked below.

🚀 Applying These Principles

If you're deciding whether TOON fits your own LLM pipeline, these are the questions its design principles suggest asking:

1. Is the data mostly uniform arrays? Tabular savings are largest for repeated objects with the same fields
2. Does a human ever need to read the payload? TOON stays scannable where minified JSON does not
3. Will the model need to generate this structure back? Explicit lengths and headers make output easier to validate
4. Does the rest of your system need to change? It shouldn't — conversion is lossless and can live only at the prompt boundary
5. What does it cost in tokens today? Measure your own JSON payload before and after conversion rather than assuming the savings